{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个实例中，我们使用tensorflow来实现一个简单的手写数字识别的网络，并用这个网络来做个\n",
    "简单的识别示例。\n",
    "\n",
    "首先导入一些用到的库。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:50.515650Z",
     "start_time": "2018-06-01T06:32:43.133728Z"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "from matplotlib import pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "tf.logging.set_verbosity(tf.logging.INFO)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T05:39:48.304594Z",
     "start_time": "2018-06-01T04:55:17.674707Z"
    }
   },
   "source": [
    "先来看看数据长什么样子"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.075054Z",
     "start_time": "2018-06-01T06:32:50.518290Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From <ipython-input-2-a34a30334354>:1: read_data_sets (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use alternatives such as official/mnist/dataset.py from tensorflow/models.\n",
      "WARNING:tensorflow:From D:\\Anaconda3\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:260: maybe_download (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please write your own downloading logic.\n",
      "WARNING:tensorflow:From D:\\Anaconda3\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:262: extract_images (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use tf.data to implement this functionality.\n",
      "Extracting ./train-images-idx3-ubyte.gz\n",
      "WARNING:tensorflow:From D:\\Anaconda3\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:267: extract_labels (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use tf.data to implement this functionality.\n",
      "Extracting ./train-labels-idx1-ubyte.gz\n",
      "WARNING:tensorflow:From D:\\Anaconda3\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:110: dense_to_one_hot (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use tf.one_hot on tensors.\n",
      "Extracting ./t10k-images-idx3-ubyte.gz\n",
      "Extracting ./t10k-labels-idx1-ubyte.gz\n",
      "WARNING:tensorflow:From D:\\Anaconda3\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:290: DataSet.__init__ (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use alternatives such as official/mnist/dataset.py from tensorflow/models.\n",
      "(55000, 784)\n",
      "(55000, 10)\n",
      "(5000, 784)\n",
      "(5000, 10)\n",
      "(10000, 784)\n",
      "(10000, 10)\n"
     ]
    }
   ],
   "source": [
    "mnist = input_data.read_data_sets(\"./\", one_hot=True)\n",
    "\n",
    "print(mnist.train.images.shape)\n",
    "print(mnist.train.labels.shape)\n",
    "\n",
    "print(mnist.validation.images.shape)\n",
    "print(mnist.validation.labels.shape)\n",
    "\n",
    "print(mnist.test.images.shape)\n",
    "print(mnist.test.labels.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T05:49:40.128071Z",
     "start_time": "2018-06-01T05:49:40.123888Z"
    }
   },
   "source": [
    "可以看到images里面有数量不等的图片，每张图片是28x28长度的一个一维向量，\n",
    "所以用的时候需要先给它还原成28x28的二维图片。labels中则是图片对应的数字的值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.695746Z",
     "start_time": "2018-06-01T06:32:51.077167Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAd0AAAHiCAYAAACtERYWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xd81EX6wPFnUggloStKr6Ep5RR7QQELYu8N9VAUbJyKenr+PM5T76yA2Cgq9nqo2LCCDQURpUgv0kF6D0l2fn8kzHxnzYbNZnc25fN+vXzdM5nZ3Tm+2Tz7ndmZUVprAQAAiZeS7A4AAFBZkHQBAPCEpAsAgCckXQAAPCHpAgDgCUkXAABPSLoAAHhC0g1QSmml1A6l1P1Rtu+nlNpe+LjWie4fSobrWbHEcD2HFLbXSqm0RPcPJVcZ36OKzTEspZQWkTZa64WF5WNF5OOwZjVE5Dyt9TuRHoeyoYjrWV9E3hORdiKSKiJzROQ2rfV3xT0OZUNx10UpdYWIvCAi12itRwd+3lxElohIutY6z09PEa2irqlSKlVEhojIX0UkS0QWisgJWuvNxT2uvODTXzG01t+ISObeslKqu4iMF5FPktUnlMp2KXgjLxARLSJnish4pdT+/EEuv5RSdUTk7yIyO9l9QVwMEZGjRORIEVkmIh1FZHdSexRHDC+XzBUi8rbWekeyO4KS01rv1lrP01qHRESJSL6I1BGRusntGUrpQREZLiLrk90RlE7hB6hBUjBi8bsuMEtrTdKtbJRS1UXkPBEZm+y+oHSUUjOk4JPz+yIyWmu9LsldQoyUUoeJyKEi8kyy+4K4OFhE8kTkPKXUGqXUfKXU9cnuVDwxvBy9c6Xgk/SkZHcEpaO17qSUqioiZ4tIlWT3B7EpnPt7SkRu1FqHlFLJ7hJKr7GI1BKRbBFpISJtROQLpdR8rfVnSe1ZnHCnG70rRORFzTfPKoTCoebXROROpVTnZPcHMRkoIjO01pOT3RHEza7C//2X1nqX1nqGiLwuIr2T2Ke4IulGQSnVRES6i8iLSe4K4i9dRFomuxOISQ8RObtwGHKNFHz55lGl1Igk9wuxm1H4vxX25obh5ehcLiLfa60XJbsjiJ1S6ggp+J2fIgVLhm4SkQYi8mMy+4WYXSkiVQPl/4nI2yIyJim9QalprRcppb4RkbuVUjdJwQfiC0Xk4uT2LH5IutHpKyIPJ7sTKLUMKfiWa0sRyRWRmSJymtZ6VVJ7hZgE122KiCil9ojIVq31liR1CfFxsRR8cNogIutE5B6t9RfJ7VL8MLzsyhGRaUqp+4I/1Fq301r/6dOzUuoqpdTmwseFPPUR0XOup9Z6kta6s9Y6S2tdV2t9vNb6672NuZ5lXpHvz7201t3DNsa4V0R+LXxchR2uLOf+dE211iu11qdorTO11i211s/urasI71F2pAIAwBPudAEA8ISkCwCAJyRdAAA88frt5V4p5zOBnCSfhd6K+3Y9XM/kScT1FOGaJhPv0Yol0vXkThcAAE9IugAAeELSBQDAE5IuAACekHQBAPCEpAsAgCckXQAAPCHpAgDgCUf7AQDKvpRUE84f1dWpmn3yUyY+/YoBJk77Ylri+1VC3OkCAOAJSRcAAE9IugAAeMKcLgCgzElr1sQpz3+wnomXdB8d1rqKiTa3snH9LxLStVLhThcAAE9IugAAeMLwMiqN1A7ZJp47oI5Tt+Ccp00cEvcI0hSxx2I+tbmFicc+1ttpV2/M5Lj0E6is0lo2N/Fvd9d36v48pGxds/xoEzf4Zr2J8+PXtbjhThcAAE9IugAAeMLwMiqUtCaNnfJv9x5g4tdOfNbEXTNCTrtQ4PNnSNy64GfT/rUXmrjhHa84rZ6bcKyJ81asjL7T+JOUqlVN3PRr5dQ91eg7E6cqe23m7NnptLv15L4mzp+3UFA2qXT7beM5/6xr4iU9Iw8nt/z8r065bf/fTBzavSCOvYs/7nQBAPCEpAsAgCckXQAAPKn0c7qrbznKKavAapGqG2xhUzv3cQdOtl9Grzp+SkL6hugsfuhIE8+99EmnLrj8J7j0JxT2efPDnbVMPGV7y4ivdUiNpSY+N3OrU7dqwiwTf9DRXZKEfQvO46583S7N+qDRK0U1FxGR7rPOMrF61F1ikrHol1L3Ka15UxPnLV1W6ufDn80b0dnES3qOitiu9cQrTdym789OXfi3MMoy7nQBAPCEpAsAgCdlcnh53fXukO/mTrkmHnfSiLi+VvsqUyPW7dZ5Jq6VUs2pW3f5DhOvGu7+Mz62ppeJN1xQ08R5y1fE3E9Edn4vu4QkfDcpd/mP/Yz55OZWTrvPTu5o4uKW+3x3+kUmPuOZp5264HKiD6Rb8Z3Gnyz8pz2YfG63JyO2a/PF1SZuO2CeiUM7ljrt3N+E6Mwf6V639056wsQXvnCLU9f0n9/H8ApYOPQIt9znqUDJvkdbfuYuC8ruP9vEsVzbsoI7XQAAPCHpAgDgCUkXAABPysyc7vxRdi5lbu9hTl2GSg+WPPUo/HVd+6fWCMRu3YvNvjbxZW90N/GmS5o67ViCUAqHHWzC6+rZudUPdx7gNAsu/5m1taGJcwbv57Rb9JC9iNn3VXfq8ufYbeWCy8PSn3UvfG5gomnlHe73Ehr9l/m/cPrIzk7560seDpTsNViW527vmN3PLs0K5e4pdT9yex5i4nG93O+MdAxsUYjY7TnF/n0fd9ZQpy5V2aVizrKgq3512ulQWTwzqOS40wUAwBOSLgAAnpSZ4eWnT3jRxOHDuv/d0MbE6/ZkxfT8/5tmh5CajlfFtIzOih7288pDvV916oI7Fb3cfKKJL3u1u9Nu04X2RByWE5XQlJkm7H/uABOnrt7oNHOX/6wx0co73F2n5hxvl4acOuoapy51jo039LO7X+XqaU674PKkZq/87vYjvP+QtXe4Q8P7p9oh5V3a1vUddKvTrnruj3Htx/a/2ffrwVXcvz3bdY6JW7y1wamrGIOdftS7e4mJO1Wp6tT1mnO6ibPvtdciv4IMJ4fjThcAAE9IugAAeFJmhpeHXnieif/RpaZTt/+7dteZ/A3u8GG0siXyzlOxaD3exqOf6+3UrXnd7pB0fe3lJg4ONYuItO1vh0Wb38Pwcqz0VDvUHO0wbtX17p42I7c0N3GVtdudusVD7DeRX7jcDkMHD1AQEZmWYz/Dcoj9vvXP/jZi3dnzzjdx9XGRh5NVmv0TpqpVi9guXP7Bdnrh8fbPR2zXfdpVJt5/9tyonx+umxp9HrFu61g7zVZ7wWQf3Ukq7nQBAPCEpAsAgCckXQAAPCkzc7p6mj1Bop67EqPMfzU/NMOd63n+8T4mvn7I0+HNjVcvsztv3XXPYfHvWCW060z333FjO/srHpzHrTfTnbftX2upibt84C73OSzDPi64LGhqjvuZ9R/97FKjVHEP2UbJZKXvNvGOsLrckw41cd17lpr4jZafluAVJhX50+/Crul+//G3A15Fs+Uye5rQcVV/MfHRM85x2tV+6QdvfSoLuNMFAMATki4AAJ6UmeFlIB5WXejucjTneDu8H1ziE37YfbAuOJwcXhdcFnT52zc47Vp+VfGXO8TTyFGnO+XrbrOHDbzY0q7Ju+77U5x2Y5rZa5omYaeNlNKV469zym0mV66hz3jafEb4xECBne+7h5Jk6sWJ60RK2O9HGdjlijtdAAA8IekCAOAJw8txsOIu9+zUUNdtUT2uQaodCs078RCnLu3LaeHNEYPgt42DnzHdnxdf13/5iSZe/nd7+AbDyaWzo3H4NbCqKXuO7dhmX4bV2iHDW9fYb6t/NKGb0yr3QPv+WnjSqKj6VP/n0h+GggIH1tla5M+rbYh83WOVc6q99uuvsecvH9RgtdNu23n29ypv9RpJBu50AQDwhKQLAIAnJF0AADyp9HO6aS2bO+WF/Q408VMXjYzqObpXdXcfSlXRfZZpnJZp4pHPD3PqBjY7JqrngKvhG1Wc8vmN7LKUg2quMvF19b532jUKHKAe/ll00YPtTVztqylx6CVERLKf/cMpt8+9PqrHtX7JnjQWmrfIxC3y3Dn2xf85MqrnG7jyaBPXfdX9LoUOb4yI0g5o4JRHtX0lUMqU0kqtXcvEZ01e4NRdmDXcxLVSIp821XHEpSZufC5zugAAVGgkXQAAPKk0w8vbzz/cxH/8xX7W+Nc5rzvtLsraFMOzl/6zS8/PBznlbPmp1M9ZGVV7zx3+zXnPxtMC16l/twFOu2332d1zvjz4DafumH/aXYl+ndbExBxUXzr58xc55RZ3LorQMuxxUT5/2s7olv/8NLqLievnsgwsZunpTrFpWumGlNcNdJdinnXtRBP3r7UqrHXkIeWg/bKK3iXLJ+50AQDwhKQLAIAnJF0AADypUHO6qmtHE9ce4W7/9VFzezJJtEt63t1h5yRm7Wocsd0HD3V3yqk5dqHBFf+yp6X8eR7CqrImPWJdZZHWxP03zlu+ImGvpafOdMqZgYNszp/knn4zrvVHJj7oaruUq+k/mdMty1Qxk795gZnhOvNzPPSm4tPb3O1vR25paOLi/val1q9n4uV/bWvimYOeimPvCmzZVdXE+8f92aPDnS4AAJ6QdAEA8KRcDy//PsT9Svk9F9mlHpdmbXDqluXZkyfm7qlj4htfu9ppV321XWZw4MT1Js7/bX7EftSSyAddL/h7YJeWsCGWJbnbTdz8ve1SGe06054SE1yaIyLywe92uuDAs+Z469OWR5o65dAzdrogt80ub/1A6Vx18YSIdecvtFMIqRN/jtgO0cvfvMUpv7bCnvzTv5Zdu3f0HT867brdZw+xvyDzi7j2acgfHZxyw5vskqG8uL5S9LjTBQDAE5IuAACelOvh5drd1jnl4JByj9/OcOpynzjAxMFdi5pL5B1oot35Jlzo+K4mPqv2mECN+xlnYyiwOf8U99u0FVnwW8oXPvixiX/a2txp53NIObiZ+nn/cYclU4SDzcuD1P32c8ptMhZGbLv+6eYmzpLkbHxf0e1+3h4ek/NwrokfPmB63F8rV9u/1h0m9TNx9t/daca835fH/bVLijtdAAA8IekCAOAJSRcAAE/K9ZxuvX7uMpvWt9iTY1oNdudq02SZlz6JiGzKtrueHF018uea/rMuM3F9ibwkqaL5/RK7JCe4lODx6T2ddq0k/nM/xmEHO8VTn//a9qm2OxcYCnw2TZ8f3Wkm8G/LCa2c8unV7dz8du3uOlV1fa4gsWq+apcA/vhvu+PecVWLar1v+Tpk4kN/usSpq/K2XQba8iX7tz9Zy4KKw50uAACekHQBAPCkXA8v5612v+rfanDZ+Or/hm5FD2rM2bPTKWc9VavIdhVdo6/sxujpN6ea+OYuXzrtxtx4monrzXaHB9O+nFbkc6d2yHbKq3rUN3Hmafb346uDX3DaBZcFhcI+i2Z/fK2Nh3xf5Osi+a4Y8n7EuiW57jVN/7zo3x/41+7by02sZmWZuMXw2U47nW+Hl/ffNjfxHUsQ7nQBAPCEpAsAgCckXQAAPCnXc7plxcmztjrlcbWfDJTsVo9XzL7CaVfn46mJ7FbZFdjy8ugZ55j4y4PfcJpdd+cTJg5JyKkbsu6QIp/6jFqvOeWuGfZxKYHPmOHPF/z82fbt652aDg/brePK4hIEFKiXGvmkrkdWnxz2k82J7Qwi6vD0QKfc/EG7La/Os++wWLfhLeu40wUAwBOSLgAAnjC8HAfn1ZzhlKunZJp4fq49NLn6iNre+lRe1L5mj4mHvO8OGT/QwP675mqnSu7b/xcTh8RWhp8IFFz+szbfHkD/1IajnHafjjjaxG3GuLuZMaRc/u0Jpe67ERLm/pZdTNxE3GV3OrxxBcedLgAAnpB0AQDwhOHlGK0baIcnG6S630Jekmu/RXnxA4NNXP9jd9gSInnLV5j419ObOHWt/1v0N5RFROZ0H23i42ZcYOI/NtaM+JjWQ+1AsZ4606mrJ1ybimxU8w+c8iGP/s3ErW79Ibw5kDDc6QIA4AlJFwAAT0i6AAB4wpxulFRGhlM+9zp7Is620B6nrveUASZu+ixzhdHKW7HSKbe6dGWEliJ9xM731pRFgTiyyrY0obK5+/VLnXK7vo/ZON19/0rIXVoG+MKdLgAAnpB0AQDwhOHlaIXcwcmXxp9g4o9/7e7UNX2TJQiAb83+z53KueX/jozYthVLxJAk3OkCAOAJSRcAAE9IugAAeMKcbpR0rrssqPndzAkBAEqGO10AADwh6QIA4InSmn16AADwgTtdAAA8IekCAOAJSRcAAE9IugAAeELSDVBKaaXUDqXU/VG276eU2l74uNaJ7h9KJobr2bPweoaUUj0T3T+UDO/PiieGazqksL1WSpXLfSZIun/WWWt9996CUmqkUmpe4R/iK4MNtdZjtNaZ3nuIkgi/nicqpX5WSm1VSi1WSvXfW6e1/rzwei5LSk8RDd6fFU/4Ne2ilJqmlNpZ+L9d9tZpre8VkY5J6WWckHT37VcRGSgiPye7IygdpVS6iIwTkWdFpJaIXCgijymlOie1YygN3p8ViFKqioi8JyIvi0gdERkrIu8V/rxCIOnug9b6Sa31FyKyO9l9QanVFZGaIvKSLjBVROaISIfkdgux4v1Z4XSXgu2Jh2qtc7TWw0VEiciJSe1VHJF0UWlordeKyGsicpVSKlUpdaSINBORb5PbMwCFOorIDO3u2jRDyvmQclC5nIgGSuE1ERktIsMKywO01suT2B8AVqaIbAn72RYRyUpCXxKCO11UGkqpdiLyhoj0FZEqUvDp+Xal1GlJ7RiAvbZLwRRQUE0R2ZaEviQESReVyUEiMk9rPUFrHdJazxORD0Xk1CT3C0CB2SLSSSmlAj/rVPjzCoGkuw9KqSpKqapSMJmfrpSqqpTi3618mi4ibQqXDSmlVCsR6SMF34BFOcT7s8KZKCL5InKTUipDKXVD4c+/TF6X4otfzn37VER2ichRIjKyMD4uqT1CTLTWi0TkryIyXES2isgkEXlHRMYks18oFd6fFYjWeo+InCUFU0CbpeD9elbhzysEkq4rR0SmKaXu2/sDrXV3rbUK+2+iiIhS6iql1ObCx4WS02UUo6jr+abW+iCtdZbWurHW+g6tdUhERCnVo/B6NpCCT9soW3h/VjxFXdPpWutDtNbVtNZ/0VpP31unlLpXCkamckSkXJ5Ly3m6AAB4wp0uAACekHQBAPDE6+YYvVLOZyw7ST4LvaX23apkuJ7Jk4jrKcI1TSbeoxVLpOvJnS4AAJ6QdAEA8ISkCwCAJyRdAAA8IekCAOAJSRcAAE9IugAAeELSBQDAE5IuAACekHQBAPCEpAsAgCckXQAAPCHpAgDgiddThsqzJa93csrfHv20iS/pe6NTl/rVz176BCCyRY8eYeKbT/nYqfvo4iNNHJox11ufsA9H2L+zS252D+mZf/xYE7eeeKVT1+qSXxLarXjiThcAAE9IugAAeMLwcpT0shpOud6x1Uy8sW2GU7ffV166hDjKOa2biTdes92pm97tlaie47oVx5r42487O3Utn11s4rzVa2LpIvYhrVFDpzzizOdN3KvaLqdu7OG9TVxvRmL7heKtGXSUiR+44TkTn1Rth9MuV9t42GGvO3XDpV2Rz732xqOccsNX7VRC/oaNJe5rPHCnCwCAJyRdAAA8YXg5SjVWqIh1B1z4u1POfybRvUEsVHoVE89/rKtT9+Hpj5u4dbo7XRCK8vmfafyNfcw1Xzt1XQ7ua+LG5zK8nAiLrm3mlMOHlJE8KsO+pzZd8Ben7uvbHjVxdVVFSmvF3+2Q8tTrhzp1b17f2MTDh57r1O33zORSv3Y0uNMFAMATki4AAJ6QdAEA8IQ53TjYlZfulEs/K4FEmPdEFxPPP/0ppy5Fqpo4JFqi0X95d6c8usmkiG2Hd7FLHB6td7yJk7VsoSJqcvSKZHcBESz+p53Hnd13RFhtdH8xn9nc0sTPvnSaU9dIvjdxTj37LYx0leq0uzRrtYm73fmYU3e53GLiRM7vcqcLAIAnJF0AADxheDlKNU9bHbFuyzvuTjj7ye8RWiLRgsuCRNwh5dl9gsNa7rDT6vydJj5u3G1OXctxe0ycscAu98lfv8Fp1/WNS008rdvLTt3Pu5qbWO/JjdB7lNTuPoeZeFjLJ8Jq0wXJE1wmVKPDphI//uOdWU75ndtPMnGjD78Pb15i2WF/K17/+yMmPrnrINvu2qmlfq0g7nQBAPCEpAsAgCckXQAAPGFOtxj53e3X3Md3fNKp+2WPnRNs8Mospy7abQMRf6uvP9Qpzz89OM9nr9mYLU2ddv+7ppeJ23z3Q8TnzyvmtXNyIs8hjl9pD+eutm1JMc+CkthVz17Tg6swh5tMKs1NJ4v+Zf9+/nZo+DKhogWX4a07153TzVgZ3dxq8w/tdzA6NbvSqZt25BgThy8napFmlw3WnJu43yXudAEA8ISkCwCAJwwvFyM/w34myVTuyTO52u5aFNq2zVufULwB/d9zyiliT4d6cEMHE08+I9tpp5b+EtXzp9asaeIVVx/k1N3e6X8mnr7HnWSodjJDysn0XY57f5G1vLiJAsQip6d7ctdvl0U3pHzzqqNNvPY0O6ybv2FVTP1I/epnEzf9yq0bN+9AE1+QuS6m5y8t7nQBAPCEpAsAgCcMLxdj6dl8Jilv8sM+RwYPL/joge4mzloa+RvKkuJ+qzH/+M4m7jPiCxNfV9sduwoOZZ8276ywJ10Z+fUQs3bXzY6q3dAVvZxylU/iu8tQZbX2Jntg/MAB70b1mOBwsojIkuPteza0s+IfAEJWAQDAE5IuAACekHQBAPCEOd1iZB3AUqCKpPqaPftuJO4crojIxy+PiupxZy/sbeKUc3c6dflRPQNKamCD4Ly6ithu3sdtnHJj+SNBParYUjq3d8r/ucnu8NSj2s7w5kZwp6ngsiCRxM7jqq4dnXLz9J8jtBRZmJtj4lqLE7ekjDtdAAA8IekCAOAJw8uoUBbsauD+oNZSEz734nAT/2dtT6fZxN9bm/iTw4aLq5qJtoR2m7jbh39zWrW71S5fCe3YEW2X4UGzd93hZIb7Y3PsS+7wbHFDykFT3z3YxI02lP4A+mjNG1DdKR+WoSO0FJmww+5YV+29KQnrE3e6AAB4QtIFAMAThpfDpFS1Zyoe0yjyJvWj1h0fKG1PYI9QEnOu7+D+4J0fTXhgqh0mHtbwO6dZSkM75BUKDCeHO+GJwSbOfsgdJuMcZT+CuyC1TQ9eg6pOu5X5gaHPPAaUY7X+2iNNPKDOo2G19iCY1fm7nJpbfre7sjX931oTJ/pKpLVoZuJJpzweVhv5vf3txtaB0vr4diqAO10AADwh6QIA4AlJFwAAT5jTDZNSu5aJn2j4ccR2k761B5i3kmJOrEHC5ZzWzcTLL3J3kkkpZpeioFQV+Pyp3dnZHrPPMXHDh/wtd0CB1Ab7O+Wul8w0cc2UquHNje7jbjNxmwW8R2O1zU6RSmZKRsR2j6w7wX3cscF50cTNkYabd709qD74PY5wmwLL/0RE1gxrZeIazOkCAFD+kXQBAPCE4eUwec0b7LuRiDT9JDfBPUFQSqd2TvmAkfZQ+NFNnjVx8ND6gnLR7lzTzSn/b8qhJn6611inbkzbl03c9wI7ZJn5JkOWXtSv4xRHN/mkyGZbw4YLs5ZwT+HTJ58f6pRbyGR/L67sNJJOje4ht6041SnXePvHCC3ji99KAAA8IekCAOAJSRcAAE+Y0w2z/u7dRf6899wznHKVib+aOPK5FSiN9f3t9nMT7nnEqavlLBWJvCzo1tVHmPjjL+2cU/bj7haf2avtqSKPnHCpUxc8xP6ie+0ysg/edOcakRj5NapE1W5mrnuizAFDWd7l04HfJW+rzS2XHm7iuRc8GdVjvv/O3TLW19JP7nQBAPCEpAsAgCcML4d5+qBXAiX73fNVW2s67RrmrfDUo8pj20VHOOXgkHKtsJ2H5uTaJVuPr+ll4nlDOzrtar37i4lb7rZLGNx9q1ypk351yu3evN7Ev54/1MTjTrrBaZf+6U/FPCtilfXo6qjaDZjuTgs0ltmJ6A4iaHbXXKe8dnx8nz+tcSMTL7i+qVP342XB048i75r12ja7JDT7+U1Ona/Bce50AQDwhKQLAIAnlX54Oa25O0yRpew3HlNVuu/uVGrrO7nfQg4OKY/bUdepe/6C00wc+uU3E2eFfQMxloPlU6q5Q9kd/7LUxBmB34lQWnSHKaDk0po0NnF25rKI7S5d2tPEza5e5dRxbL1fx9Re6JTfbWOni/IXLI7qOVLbtzHxgivqO3VDz3vexCdV2xH2yMhDykFjrz/TxGmzp0X1mHjjThcAAE9IugAAeELSBQDAk0o/p7t7tFvOTrfzefmBw8wz33SXDCHxggfQ3/HVBU5d9i9T4/paqfXrmbj6OHeu9o2WHwVKzOP6sKZ3ExO/v//7Tl2qsvcKm3bbXahS9rhLQFS63clK5+6JdxcrjTaj7ZKtIb27OHX37meX5F1Vc7lTl/q+/fs5c2djiUaXGpNMfGlWdEvFwr2/w+4Ud9vnFzl17X6wy8hi+b5HPHCnCwCAJyRdAAA8qZTDy6nZrUx8a/P3I7a7eInd6ajm634OOK7M6s9wj47YFNpl4qm9hzp13Z4dZOL2//e7ifPXrov4/GmNGpp4R+dGTt2gYa+Z+LTqW5y64DDUk5vt7061b+ZGbIfECU77fNQu8P6d77Zr8/ZAG9/sZzP7iihv8VITTxh+jFM3aIj9dw3fNa5vzZW2EIzjYKd2pwue3GiHvb/+azcTZ/80xWlXFt6j3OkCAOAJSRcAAE9IugAAeFIp53T3NKpl4h7VciK2m/9GWxM30ByInWhZr7vzbse1HmziXwc84dTN7/OMiWefZM8MGrTgwojP/0p7e4JU+PxTcHlS+LzPravtdnZzb7QHX6ttvwoSo+pGexUW5e1y6lqlVSvyMbvC5vmqr+aeIt7qPjfZKf/fgB4mvm6/iU5d+/T4bqMb/D7FS8NOderqjwz2a1ZcXzfe+K0EAMATki4AAJ5UyuHl4ly34lgTN3xtnok5scQQIOiAAAAgAElEQVS/unPtv/ozm1s6dR2qrjBx96p2aPizju8U84xVI9Y8s6WZiR//sI9T1+ae6SZWuxlS9iHzLbtE74IDBjt1v/z9KRP/e307E78z8kSnXaMRTAkl2qJuu018Z+uL3borDzDxyaf8ZOJHD3SnkTq+eIOJVTF/aFu9usHE9X+bHLlhGcedLgAAnpB0AQDwRGmt990qTnqlnO/vxeD4LPRW3HfqT+b1TGve1MQL/lM7YrsH//Kuib/f1trE4ycc7rRrcVf5Gq5KxPUU4T2aTBXtPVrZRbqe3OkCAOAJSRcAAE9IugAAeMKSIZRLeUuXmbjFRcsithspwaVGdpejFlK+5nABVAzc6QIA4AlJFwAAT0i6AAB4QtIFAMATki4AAJ6QdAEA8ISkCwCAJyRdAAA8IekCAOCJ11OGAACozLjTBQDAE5IuAACekHQBAPCEpBuglNJKqR1KqfujbN9PKbW98HGtE90/lEwM17Nn4fUMKaV6Jrp/KJkYrueQwvZaKcWJamVQZfybS9L9s85a67v3FpRSpyulZhVe6O+VUh321mmtx2itM5PTTUQp/HqeqJT6WSm1VSm1WCnVf2+d1vrzwusZ+axAJFv49eyilJqmlNpZ+L9d9tZpre8VkY5J6SVKwlxTpVR9pdR3SqkNSqnNSqnJSqmj9zasCH9zSbrFUEq1EZFXROQ6EaktIuNF5H0+NZdPSql0ERknIs+KSC0RuVBEHlNKdU5qxxATpVQVEXlPRF4WkToiMlZE3iv8Ocqn7SLyVxHZTwqu6X9FZHxF+ptL0i3eySLyjdb6W611nhT8AjQSkeOT2y3EqK6I1BSRl3SBqSIyR0Q6FP8wlFHdRSRNRIZqrXO01sNFRInIiUntFWKmtd6ttZ6ntQ5JwbXMl4LkWze5PYsfkm7xVOF/4eWDktMdlIbWeq2IvCYiVymlUpVSR4pIMxH5Nrk9Q4w6isgM7W42MEMYUi73lFIzRGS3iLwvIqO11uuS3KW4IekW7zMROV4p1b1wyOouEakiItWT2y2Uwmsi8n8ikiMi34jI3Vrr5cntEmKUKSJbwn62RUSyktAXxJHWupMUjEpdIhXsQzFJtxha67kicoWIjBCR1SJSX0R+E5EVyewXYqOUaicib4hIXyn48NRRRG5XSp2W1I4hVtul4A9zUE0R2ZaEviDOCoeaXxOROyvS9y5IuvugtX5ba32Q1rqeiNwrBcORU5PcLcTmIBGZp7WeoLUOaa3niciHInJqkvuF2MwWkU5KqeAUUKfCn6PiSBeRlsnuRLyQdPdBKXVI4fzfflLwrdfxhXfAKH+mi0ibwmVDSinVSkT6iMivSe4XYjNRCr5oc5NSKkMpdUPhz79MXpdQGkqpI5RSxyilqiilqiml7hCRBiLyY7L7Fi8k3X0bJiKbRWRe4f9ek9zuIFZa60VSsBxhuIhsFZFJIvKOiIxJZr8QG631HhE5SwqmCzZLwbU9q/DnKJ8yRORJEdkgIitFpLeInKa1XpXUXsURpwwFKKV2S8EXbIZrre+Jov1VIvK4iFQVkQ5a68UJ7iJKIIbr2UMKknCGiPTWWn+V4C6iBGK4nveKyC1ScD1raK3zE9xFlFBl/JtL0gUAwBOGlwEA8ISkCwCAJyRdAAA88bqJdK+U85lATpLPQm+pfbcqGa5n8iTieopwTZOJ92jFEul6cqcLAIAnJF0AADwh6QIA4AlJFwAAT0i6AAB4QtIFAMATki4AAJ6QdAEA8ISkCwCAJyRdAAA8IekCAOAJSRcAAE+8HngA+JbWrImJNx/eyMSr++xx2g34yyQTD6oz36k76NurTBxaWsPErYf86rQL7dwZuR8HHmDivNVr9tVtoELJ63GIiTd0zHDqdu1vz2TQrXeY+I7Onzrt+tWy75tPdrrPMXhkPxM3fOj70nU2wbjTBQDAE5IuAACeMLyMCmXV4KOc8t1Xv2biszPXRXxcSuDzZ0hCTt2MY8bYwjE27Lz7Zqdds3sjD2tlvJFv4rzjIjbDXsoeRbpuwJFO1YAb3zVx/1qrYnr6kVsamvjdM44wcWjpCqedznWnIRC9LZfZf9cv/zPcxBnKTTshKfrI3xRxj6PN1bZdj2ruVM63Nz1q4qNSbzVx4wfL3lAzd7oAAHhC0gUAwBOGl8OkdG5v4nm3VDPx5V1+dNrdWHeKiXs8OtipO2Bo2RvSqMhSO2SbODicLBJ5SPmP/Byn/HtedRPnS7pTd2gVO8SYGhj2/PXqYU67blvtcPOBj7q/A8fUXWTiCVKzyD5VeimpJlx+9+EmnnndiIgPydF22H5VnntNqwZGJ/dPre7U9atph5H7TXzbxMM2tXbafdHnIBPnLV0WsR/4s61nbTdxurLXNnw4eVneLhPfveKMiM/349yW9vlquMP+3x79tImPOsuuKlj+mPstZ53j/o4kA3e6AAB4QtIFAMATki4AAJ5UyjldlWHH+df0P8Sp+/FOO0+3LWTnDY54/Tan3ddd7NzP8ZdNdermDY1LNxGluXdmmjh8Djd4DU/46RoTNxhW1WmXOvHniM+//lq7ZKXPwK9NfFf9X5x2+e70kePbja0CpT8iN6zEVg6Odh43z8SdX7Xz6C1vn+y0S23fxsRz/57l1M068RkTB5ew3FxnoftiH9jw8+4tnKr89Rsi9hEiza9ZaeKBn9h1crM2HuC0qxNYeZc/f5FEki0bI9Yd/szfTDz/dDu/2+XWG512jR9I/vdtuNMFAMATki4AAJ5UmuHllKp2OHHu0E4mXni6O4z1xGY7JPXWkFNM3OrNsKGrbDtcOKNVF6dOn27XKqTttEsa0r6YVtJuIwr/O/bpQMn9HDnwd7sEoeHZv8X0/PWftdf+y3V2S6q7RvxSVPMizfvE/l41ZnhZRERUmvvnp8rR0Q3XHvQ/O2TYJmxIOSh/zgLbrq9bd2x/O6b50B0jTdy9aq7TLjjc/EXWwe6TMLxcrPxNm0w8fZSdoqm9yF22kz8/8tROtFJ3FH3/2LH3PKe85YFSv1SpcacLAIAnJF0AADwh6QIA4EmFndNNqe5u+7by1WYmXtjNLhd4bFMbp92EG483ceZXP0R8/uBX26tv2urUDZo80cSj19ivym/5Yh+dRkwOrmK3bQzfYm7qfLvMI1tKPweXNcvOx3672112VG92XnhzQ6uIVZVWatPGTnnqIa8V2e6JzS2dcrtn7FxhfnjjKNUfaeeCx11zqIm7N4w8R4zY1RudnH/XPvV/dcqvSOMILf3hThcAAE9IugAAeFKhhpeDQ8pzHz3IqQsOKT+ysa2Jvz6jg9MudUnJv76+/Ep3iLpHtQkm3riffb4Xa3dy2uVv3lLi18KfnTDrXBN/dtCbTt3Y7qNNfL+4S7uildfD7lq23312WqFlmnv96t+6xMQ73nOfQxV9TneltvTChhHrtmu7rOT1B05x6mr9FnnaJxaLr2xu4u/Gu6eJHZ0RMvGC/m5/W95jd1zSeZGnFhB/Oad2c8pX9ppYZLt313UN+0nyl+txpwsAgCckXQAAPKlQw8t/XNrZxAvPeNKp+3Cn3RT/6zM7mjhvydJSv+6eWpHHDufstkNSDCcnRuYg+2v89NvuUH//WvNNPP+pw0zc4b+rnXZrT7Lfajz9hklOXd/a9hCMhmnBUw3cEw5ebDnexH16uxut51VjfFlEJLVeXRPfccWbEdu9vc1+67zWK/EdTg6XP9vuWnTFhP5O3cIz7LTUnL7u35TT3glsc/XTrMR0rhJLrVnTKa+92P7dvnaQO3/Tr+YKEy/N22XiDQ+7h1RUZXgZAIDKg6QLAIAnJF0AADwp13O6aY3cr/DfPvhVE6/M3+nUPXjvQBPXXFz6OaK0ls1N3OfUHyM3RMIFT5N5adipTt2Ae23d3DMDc3Jnus+REvj8GZKQWylFn05/x5ojnfL4r+3ORu1mrnDqrn3InnA04R53rqoyUYHTvi7NWpfEnhSt5tywP4lnFN1ORGTedfb/S/bVCepQBZTSxV2muap7bRNvbWuXXl1ztPvdisH1virmWe2Wbz0/usXE2eOnxNjLxOFOFwAAT0i6AAB4Uq6Hl0P13GG6c2vYjdD/tf5wp67mqyUfUg4esr1y0GFO3Z3XvGHiizKT/zX0ymzXmfbaHHvt1Lg/f7/fe5n4j1uamjhlxkKnXeud9neM/YlK56tN7QKlzUnrB6KXduABTvmKSfaQg5OrrzFxurhDvukqtdSvfcxtdvow+434/w2IJ+50AQDwhKQLAIAn5Xp4uThn1JzulD/of7OJ03dG3h1o42l2N5MPjnrKxK3S3CGRd3fYb9y1fv86py64i83Ujc0CNauK7zSitvEq+83hC2791MSD6swPaxnd58rgEFeHJ93dpJrc/32gZIc6w7/jXJwUVZLWFdfiq5tH1W7W6/Ybrg3k+2JaoqzQddzpvrNrbAyUqiT0tZ0DRUKxnrLsB3e6AAB4QtIFAMATki4AAJ6U6znd0Mx5Tjn7Tfu18fkXPOXUTbnXPSEkGp/sqmfis0b/1alr+tA0E7dru9V9YGAXmwVT7ZxuS+Z0Y5bWrIlTvueusSY+tfo2E4fvJrUx3x6GfsYMew1fPOgFp13rdLvrVNruUnW1SCHN51sRkd3N9iS7C0iU1e7SycOnXWLirvuvNPE3Xx7stKu2VklRdjVwv3vzr3NfN/G5meudut53TTTxR9LdxFmvJ/aEqljwlwAAAE9IugAAeFKuh5dFu8MPrf9mhxIOm3u9UxfqvUmKsnldllNu/o6Nq3xidzZpErZsIfjKesZcp+7f6w8y8WUn2027v789sV+br2hS27Y28YMTXnbq2qbbJT7L8uwQcu+XBzvtWj/1u4nrrrTLifq85P5+zD1xtG13ctg0wOOBHXNiXI4w5tVTTNyYJTCogPI3uX9j9zvDloPHf7SQyRKLl56wuww+8Xx1p+7Lg+0OgZOuaWMr3gzb7aoMLCfiThcAAE9IugAAeELSBQDAk/I9p1uM+s+GzRs8W3S7/ePwWqn16jrlrtXt3PK0nS3i8AqV04J7M00cnMMVEfl8l52L/+f9N5m4+fPudY902k/ry91tQs+ddJqJJ3R8y6k7YqDdQnT/EbHNxzZ+gHncfVmdv9PENZeV/XOaaizkOxo+5a22JxVlnuLW3Tr1GBN/1O5dEx9xzQ1Ouz/lhSTgThcAAE9IugAAeFJhh5d90o3cQerTqm838c3f2NNwsuUnb32qCF444rmIdQ/ffLmJ635Y+iGjRZ+0tAV3REquHjjexO+PqCdIjKwUO4WQU9PG1RL8uqnt7RKTy66ZEPXjmo1dbOKyPxgeH6l16jhlvcfuMBbascN3d4xPvu5q4scvslM5Z1//ldPum2ereutTJNzpAgDgCUkXAABPGF6Og5W96kasS1uf7rEnFUtqYN+vlLDPhxkbcsKbl0rzF+xQ4ct93cMVjq620MQf1s82cf76DXHtQ2WQNTvwjd+T3bpMZQ+dOPJmuxvcnBcT26dGL9gdyG6psyBiu/Zj3V3MWv4xNULLiiWtSWMTd3hvpVP3wXt2+qzpkMR+Q19l2N+PZYMPcepu7/1uePOCPlVZH/aTxkW284k7XQAAPCHpAgDgCUkXAABPmNONg5w6et+NUGIvbzjKxF0bfuvULf2bjVs+2MHEoV9+i+m1dJ49fWRLvnuCSfsq9rPpurPtnG69UdEvVdp20REmLosHa/vS5PWltnBL5HYHV7fn0syRA+Lej8X/sXORbzZ6LFCT4bQbtcXO77d+fKFTl59XORYKbTmskYn/0+B9p+6uq78z8SH1/+bUtR29tcSvtfj82ibOrRNy6u7r+baJL8h0549TRJk4+Kin7jvPaVdLkv/e404XAABPSLoAAHjC8DLKrE8//4st9HWHl2ccM8bEq96zy4ceXdfDaffxN10lGuPOGWri8MMVpufYz6b7vfKrid3Br+Kd949PTTzh9ZoleGTFogO7Fg3b1Nqpu7mOHb69OGuZie9/sbfTru0j9mCE0Iy5Ub3u9vMPd8rTL3vcxNUCS5WCw8kiIu+fa6c48v+IvJyoIquxcpeJ/73+IKfuH/VnmXjeOU85dSnnBId8g8v/lNMuWOc8Psp2IiLrAodlHP3erSbOfts92KQsTARypwsAgCckXQAAPCHpAgDgCXO6CZCq7GeZOrOT2JFyrvXQRSb+8UJ3O83DM3JN3DjNnkPzaNjSokcvdMuRpIh9/lDYbO3H2zrZup07JRaj5hxt4qYyM6bnqAjyN28x8Rd93PlB+cCGwfndBT1GO81eOswuIfrv6+6SkKBLz/nSxrUedeqqqerhzUVE5ImXz3TKjeckdmvDcuGHGSb8+pYjnaqT/m7n5f/X7g2nLritZ/j8bJC73MfOur6yzT297bxMu11nx08GOnXNxtnnaPPhjyYuC3O44bjTBQDAE5IuAACeMLycAPnaDk/WmbO9mJYoTv7adSb+zynnOnXzBu5n4v49vjDxoLqx7UjVb9kJJp46wR32bDlmWaC0QmLR9PzKO6QcSd7SZU751WGBY4duDoR13J2gLs9aY+NrRkT5au5w8gtbG5r4nfOON3HjOT8KIkv7Ypr7A/vWkzNOv9mpWnWxPeB+yrF2OdF58y5y2q3/wJ78owIzOw1fcZeDje1sh/6zv/wp6j6XNdzpAgDgCUkXAABPGF5OgOC3lxEf+fMXOeXWg2z5S6kRiLvF+Ap2c/am4n5jtXJsa598wQMkPn2hvok/b97FaTf3Bvut1mMOs9MJ307pIJG0G7nJKYfmLzGxzp1X8s7iT6qOn+KUW4638UVid/ZKE3da4YCw8l75YeW0LzeWqn9lBdkBAABPSLoAAHhC0gUAwBPmdBNgUa5dJpS62e5gFD5HAaBoOtcuN8lfsNipa3OzLa8N/ryYA8p576Gs4E4XAABPSLoAAHjC8HIcNP/HZKc88B/HBEruUhcAQOXFnS4AAJ6QdAEA8ISkCwCAJyRdAAA8IekCAOAJSRcAAE+U1jrZfQAAoFLgThcAAE9IugAAeELSBQDAE5IuAACekHQDlFJaKbVDKXV/lO37KaW2Fz6udaL7h5LhelYsXM+KJ4ZrOqSwvVZKlcuzA/j2coBSSotIG631wsDPRorI8SLSRkT+qrV+IZrHIfnCr4tSKltEHhaRo0QkVUSmishNWut5xT0OZUMR1/NYEfk4rFkNETlPa/1OpMeh7IjwN7eLiIwRkfYiMkdE+mmtfwnUNxeRJSKSrrXO89rhOOBOd99+FZGBIvJzsjuCUqstIu+LSFsRaSAiU0TkvaT2CDHTWn+jtc7c+5+I9BGR7SLySZK7hhgppapIwXvyZRGpIyJjReS9wp9XCCTdfdBaP6m1/kJEdie7LygdrfUUrfUYrfVGrXWuiDwuIm2VUvWS3TfExRUi8rbWekeyO4KYdZeCI2eHaq1ztNbDRUSJyIlJ7VUckXRRmR0nImu01huS3RGUjlKquoicJwV3Rii/OorIDO3Oe84o/HmFQNJFpaSUaiwiT4rILcnuC+LiXBFZLyKTkt0RlEqmiGwJ+9kWEclKQl8SgqSLSkcptZ+IfCoiT2mtX0t2fxAXV4jIi5pvhpZ320WkZtjPaorItiT0JSFIuqhUlFJ1pCDhvq+1jmqZAso2pVQTKZgLfDHJXUHpzRaRTkopFfhZp8KfVwgk3X1QSlVRSlWVgsn8dKVUVaUU/27lkFKqpohMEJHvtNZ3Jrs/iJvLReR7rfWiZHcEpTZRRPJF5CalVIZS6obCn3+ZvC7FF8lj3z4VkV1SsLZzZGF8XFJ7hFidLSLdROSqwk0T9v7XNNkdQ6n0Fb5AVSForfeIyFlScE03i8hfReSswp9XCCRdV46ITFNK3bf3B1rr7lprFfbfRBERpdRVSqnNhY8LJafLKIZzPbXWYwuvX43g+k6t9TIRrmc58Kf3p4iI1rqd1npMeGOuZ7lQ1N/c6VrrQ7TW1bTWf9FaT99bp5S6Vwr2TsgRkXI5f8+OVAAAeMKdLgAAnpB0AQDwxOspDb1SzmcsO0k+C72l9t2qZLieyZOI6ynCNU0m3qMVS6TryZ0uAACekHQBAPCEpAsAgCckXQAAPCHpAgDgCUkXAABPSLoAAHhC0gUAwBOSLgAAnpB0AQDwhKQLAIAnJF0AADwh6QIA4InXU4bKmwVj/2LieT1HOXUn3jDQxNXH/eitTwBQGaR2bOuUl55Tz8SH9p7l1L3Y7GsT5+r8qJ6/x/UDnHK1d6eUtIsx4U4XAABPSLoAAHjC8HJxtD2DOCQhp2plDxu3GeerQ9grrUUzEy8/u5GJt2XnOe3aZq808fi275s4+4PrnHaNJ9jPnzWnr3Hq9PadJs7/4w8TqzT37bPqpsNMnFfN7W/TR6bZ58vJEQB/tvWSI0x82p0Tnbpx9WZGfFyutu/f8L/VkTw9dJhTHjyvr4nz5yyI6jliwZ0uAACekHQBAPCE4eUYtWq/ysQqI8OpY/gw/tYMOsop/zT4CRNHO5wUbDW/zzNuXZ/Iz/HGtgNN/NzfzjbxqmPdt8/MK9zhqqDTJ15jYvXdL/vqKlBhpVSt6pQX/bOriWdfPsLE0b6vY5WdXsUpz7m5jq27Lrx1/HCnCwCAJyRdAAA8IekCAOAJc7ox+qjduyY+M7OXU5fPnG5cpLZuYeKxNz8eVlvyX91x2/c38bmZ66N+3IVZq208+ikTp4R9Zg3OQE3PcetSt+wusl1ls/YmOze/9dDdxbRMrPQMu7Rs1jHPR2zXp9EhPrpT8Sm7/DI4hysiMvPy4YFS6e8DO7x5Y8S63y54ImLdgye8ZeLnD+tjK6ZEXqoUC+50AQDwhKQLAIAnDC+jzFrV2y7VaV8l8ufDE2deaOIa99WM2C599WYTj2lY26nLqWeXDwx86C2n7uzMdfvurIjM2qNNPPjWgU5d9VkciiEisuMIu7vXnONHRWwXHLqPdelItM8RrHl5a5OYXgt/FjrWDiMv7m9//tuJw4to/Wdvbz/AKf/jW7tcr8n77t+Dau/Zwwpayw8mVl07uk96QeTXC77Ph7esYeKsOJ+DwJ0uAACekHQBAPCEpAsAgCfM6aLMOubyaRHrVufvMvHamQ1MnHpq5Odr8JOdt117aKr7Wj3tsoBo53DDfbC1i4mrj2MOtyhtBi4x8TlZZzt1S65sauKcOnamVWmJSaj+HhPP6flsxHbtPrLz7+1vXxhWuym2F6+MAsuCRMLncUdG9RSnzzvDxKF79nPqsr/7Kfa+lSHc6QIA4AlJFwAATxheRpn14U+dTfzQ6d84dU3TMk0855IREpWrbJiu3OHlXJ0fKLmfRdcHhrKPfes2E088/xGn3V317RB19wuud+oy3/xBIJK/eYstBGMRaXLfiri+1vYL7IHo0tOtW5hrd6Rq//BG279NDCeXRPDEoPCdpqJdGvRjTrqJ9YkrTaxkZVHNyz3udAEA8ISkCwCAJwwvo8zKHmC3gvlLvX5O3cyjXzBxLDsW5YZ9I/b9HfYA62FLejh1KcPqm7jVR3aY+Ngatzjt5p7+pIlX9cp36rLfLHEXUUqr++yJWDdkhd3QPn/+Ih/dqZB0+1Ymdg8uiKz9F9c65VYj7fs3RX6JT8fKMO50AQDwhKQLAIAnJF0AADxhThflQssh7vxc947XR2gZm9o/rTFxtcVLwmrDy/t2cPZyp5wTS6dQKgt6jDZxKOz+YtqUNiZuLRu89amiWdmjlolTirmHG7ejronbjMh1K+N8SHxxgn3887JBG2t3c6049wEAAHhB0gUAwBOGl1Eu5M+e55QzZ8f3+fP23eRP2mZH3jFn5nz3MPRsWROhJRIlJDoQu8vKYj1EobJLa9LYKZ9yyWQTF7d0744vLzRx9pQ4nwpfjBX3uOVgH8OXDV6x1G5bVufD30zsLv4rPe50AQDwhKQLAIAnDC8XJzAGFf7NvPBvvqFyyO15iIkntHXPCJ0c2Li97VM7nTpGMxNv15mHhf0k8nnM+XXtN2gXv2rPQT6k2TKn3aADP7OPEfcrrdc8d4OJm/z7+5J0tdzaeKw7vPzvBuMitu016wITt799ronjPVwbbukbnUz8XJcXon7comfambj21snFtCwd7nQBAPCEpAsAgCckXQAAPGFOtziBbUnCvw4f/Lr5nAdaOXXZ124UVBwpWVkmfmCknccNn9f/erudE9LT47ymqQJKbbC/U952VAsT76pr7wdSzlkf1fON7Tg07CcZEdvOPemZqJ6z3++9TDztkw5OXfPH7Ik4JT/nqnzacMbOfTcqtHxFPRNnby35rm6xur3TpyY+NCPyDHK/ZSc45XqfLDRxIuedudMFAMATki4AAJ4wvBwPVSrL4FLlkFqvrlPe/qrd1L1rRuQdbZ6bdLyJ28iPielcOZd70qEmzrpnqVM3ruUIEweX6BW305Erfd9NCgWHjf+4pWnkhj/MMGFTcZcFVcZ3/V1dPnHKxR1ykN3vp0R3x9j6sZ3i61szuFQscv9+e66jU673R+KWCQVxpwsAgCckXQAAPCHpAgDgCXO6QJjl/do55Z8OGlZku3+v7+SU2z++1sSxnFpUGfx+qv2TM6HlBKfulW2NTLw5v7qJ31vV2Wm37qtGUpTh/Z51yj2q2YUf3X6+2Kmr22d+oLS5+E7DyNfufVr08+2ll1rbfrdi4TPNnLrZnZ6Pqk8d3rzRxK1H+ZnDDcedLgAAnpB0AQDwhOFlVErBXaZERNa+0tDE73R+OKx1FRMN32SHnic8dKzTqtbiH+LXwQqq9hy7y1v2R9c5de0H2yHf/M1bTFxFfnfaNQ4r7/XrJe6Q43FVF8TcTyRf8EQvEZEG99nrOa7pmLDWRd8/fr7LfZ+3HWV3C0z0aUeRcKcLAIAnJF0AADxheBll1ppBR5m4Sk930/tHO7xp4pCO7rPj/UtPM/GQFu86dcGdpoLDyeG+uryOiFwAAAQBSURBVNDuqFRrNsPJJVV/5ORA7NYlcrgv/eW6+26EuNpw9ZEmrjc6um8Kz3/eDik3a7TBqRvV9IsS9+HGj69wym1+S/5OcdzpAgDgCUkXAABPSLoAAHjCnC7KjG0XHuGUfxr8RMS2wQPkc3VuVM//UTs7jxt+AH3wxKAtod1OXY9HBpv4gNnuSTNIrtQG+5u4YXrRS4lERNJ2V8YzgeLvoRknOeW+xzwfoaVIh36zTfzTgfb7Gf0v+shpd33tRSZOV7+YOFeHz/JHvkcMvp+zx95g4jZ/T86uU8XhThcAAE9IugAAeMLwchiVZv9JMmrsSWJPKp/VvdxjAorbuDw4HBzLpuvhB9AHn+P/1vRw6hp9+oeJk7WLDYq27agWJj4788OwWu4p4q3xyHSnPLmbHdY9PMOd5nGW+FwXeblP8N0b7fs6uDOciMio8XbYu+U/fzZx2Nu8TOC3EgAAT0i6AAB4QtIFAMAT5nTDpLRoauJfjnouYrvgspKGH/HPGKvUenZ7vosPmZLEnliPN/zGKX81PtPETxzT3cR5a9YKyo6UsHuI4Hs0bTuz8fGQ9sU0p3zbkAEm/uaB4XF9rRV5OU75obW9TLz8yiZOXYvf7NKgsjiPG8SdLgAAnpB0AQDwhHHRcBs3m/DgF28y8aHHzXWarXi4jYkz303+yRXlVW4He/D4vftPiPvzn/LbeSZeO6mRrVBuuzsvtacWXZi12qk7odp2Ez+REfkEIiRX+BKTF7ccbOL0z6eFN0cc1P/E7ibVtcnNTt30AcNK9dxnD7vdKR/4WHA3uPmleu5k4k4XAABPSLoAAHjC8HKY/A0bTdwisFn2hrB21aRsfNO2vEtfbYfzj5l+qVP3bddXIj5udf4uE/d6yR5I0HrkCqddxio7VNwkN/KG+K8/09XEb1Q/0qnbekhDE2dtLb/DWpXNqDlHm7ipzExiTyqu/LXrTNzk3+ucujP+3a1Uz32gVMzDRbjTBQDAE5IuAACekHQBAPCEOV0kVf7CJSau28etO0OimxNqLnbuPa+YdsX2448/ItZV/325bRfj8yMxVvaMXJf5UWbkSiBJuNMFAMATki4AAJ4wvAyg3ErZbbcWe3TDQU5d3ecnhzcHko47XQAAPCHpAgDgCUkXAABPmNMFUG61uvUHE0+SaknsCRAd7nQBAPCEpAsAgCdKa53sPgAAUClwpwsAgCckXQAAPCHpAgDgCUkXAABPSLoAAHhC0gUAwBOSLgAAnpB0AQDwhKQLAIAnJF0AADwh6QIA4AlJFwAAT0i6AAB4QtIFAMATki4AAJ6QdAEA8ISkCwCAJyRdAAA8IekCAOAJSRcAAE9IugAAeELSBQDAE5IuAACe/D8gDsctzBrBmwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x576 with 16 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(8, 8))\n",
    "\n",
    "for idx in range(16):\n",
    "    plt.subplot(4, 4, idx + 1)\n",
    "    plt.axis('off')\n",
    "    plt.title('[{}]'.format(np.argmax(mnist.train.labels[idx])))\n",
    "    plt.imshow(mnist.train.images[idx].reshape((28, 28)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，定义用于训练的网络，首先定义网络的输入。\n",
    "\n",
    "这里我们直接使用上面的数据作为输入，所以定义两个placeholder分别用于图像和lable数据，另外，定义一个bool类型的变量用于标识当前网络是否正在训练。\n",
    "\n",
    "为了让网络更高效的运行，多个数据会被组织成一个batch送入网络，两个placeholder的第一个维度就是batchsize，因为我们这里还没有确定batchsize，所以第一个维度留空。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.706044Z",
     "start_time": "2018-06-01T06:32:51.698913Z"
    }
   },
   "outputs": [],
   "source": [
    "x = tf.placeholder(\"float\", [None, 784], name='x')\n",
    "y = tf.placeholder(\"float\", [None, 10], name='y')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "因为我们输入的是图片展开后的一维向量，所以第一步就需要先把一维向量还原为二维的图片。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.719298Z",
     "start_time": "2018-06-01T06:32:51.707730Z"
    }
   },
   "outputs": [],
   "source": [
    "x_image = tf.reshape(x, [-1, 28, 28, 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:07:36.530623Z",
     "start_time": "2018-06-01T06:07:36.522665Z"
    }
   },
   "source": [
    "接下来，我们定义第一个卷积层，使用6个5X5的卷积核对输入数据进行卷积，\n",
    "padding方式选择valid，所以输出数据的宽高变为24x24,但是深度已经从原来的1变成了6。\n",
    "本层卷积的激活函数为relu。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.764292Z",
     "start_time": "2018-06-01T06:32:51.721295Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From D:\\Anaconda3\\lib\\site-packages\\tensorflow\\python\\framework\\op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Colocations handled automatically by placer.\n"
     ]
    }
   ],
   "source": [
    "with tf.name_scope('conv1'):\n",
    "    C1 = tf.contrib.slim.conv2d(\n",
    "        x_image, 6, [5, 5], padding='VALID', activation_fn=tf.nn.relu)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来进行stride为2的最大池化，池化后，输出深度不变，但是长宽减半，所以输出变成了12x12,深度6."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.774112Z",
     "start_time": "2018-06-01T06:32:51.766784Z"
    }
   },
   "outputs": [],
   "source": [
    "with tf.name_scope('pool1'):\n",
    "    S2 = tf.contrib.slim.max_pool2d(C1, [2, 2], stride=[2, 2], padding='VALID')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:10:16.678485Z",
     "start_time": "2018-06-01T06:10:16.671472Z"
    }
   },
   "source": [
    "接下来，我们定义第二个卷积层，使用16个5X5的卷积核对输入数据进行卷积，\n",
    "padding方式还是选择valid，输出8x8,深度为16，本层卷积的激活函数为relu。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.805912Z",
     "start_time": "2018-06-01T06:32:51.776959Z"
    }
   },
   "outputs": [],
   "source": [
    "with tf.name_scope('conv2'):\n",
    "    C3 = tf.contrib.slim.conv2d(\n",
    "        S2, 16, [5, 5], padding='VALID', activation_fn=tf.nn.relu)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "再进行一次stride为2的最大池化，输出为4x4,深度16。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.814748Z",
     "start_time": "2018-06-01T06:32:51.807560Z"
    }
   },
   "outputs": [],
   "source": [
    "with tf.name_scope('pool2'):\n",
    "    S4 = tf.contrib.slim.max_pool2d(C3, [2, 2], stride=[2, 2], padding='VALID')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "池化后的数据是3维的，这里做一个拉平的操作，将3维数据展开到1维，然后送入两层全连接，全连接隐层中神经元个数分别为120，84。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.856740Z",
     "start_time": "2018-06-01T06:32:51.817287Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From D:\\Anaconda3\\lib\\site-packages\\tensorflow\\contrib\\layers\\python\\layers\\layers.py:1624: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use keras.layers.flatten instead.\n"
     ]
    }
   ],
   "source": [
    "with tf.name_scope('fc1'):\n",
    "    S4_flat = tf.contrib.slim.flatten(S4)\n",
    "    C5 = tf.contrib.slim.fully_connected(\n",
    "        S4_flat, 120, activation_fn=tf.nn.relu)\n",
    "\n",
    "with tf.name_scope('fc2'):\n",
    "    F6 = tf.contrib.slim.fully_connected(C5, 84, activation_fn=tf.nn.relu)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:17:39.880958Z",
     "start_time": "2018-06-01T06:17:39.869797Z"
    }
   },
   "source": [
    "###### 对特征添加一个0.6的dropout，以40%的概率丢弃特征中的某些数据，\n",
    "这样可以提高网络的推广能力，减少过拟合的可能性。\n",
    "\n",
    "需要注意的是，dropout仅在训练的时候使用，验证的时候，需要关闭dropout，\n",
    "所以验证时候的keep_prob是1.0。\n",
    "\n",
    "dropout的输出最终送入一个隐层为10的全连接层，这个全连接层即为最后的分类器。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.913419Z",
     "start_time": "2018-06-01T06:32:51.860766Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From <ipython-input-11-413a88f1eb86>:3: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.\n"
     ]
    }
   ],
   "source": [
    "with tf.name_scope('dropout'):\n",
    "    keep_prob = tf.placeholder(name='keep_prob', dtype=tf.float32)\n",
    "    F6_drop = tf.nn.dropout(F6, keep_prob)\n",
    "\n",
    "with tf.name_scope('fc3'):\n",
    "    logits = tf.contrib.slim.fully_connected(F6_drop, 10, activation_fn=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来定义loss和用于优化网络的优化器。loss计算使用了sparse_softmax_cross_entropy_with_logits,\n",
    "这样做的好处是labels可以不用手动做one_hot省了一些麻烦。这里使用了sgd优化器，学习率为0.3。\n",
    "\n",
    ">试试看，增大减小学习率，换个优化器再进行训练会发生什么。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:52.084738Z",
     "start_time": "2018-06-01T06:32:51.915376Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From <ipython-input-12-3c77ec77605a>:2: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "\n",
      "Future major versions of TensorFlow will allow gradients to flow\n",
      "into the labels input on backprop by default.\n",
      "\n",
      "See `tf.nn.softmax_cross_entropy_with_logits_v2`.\n",
      "\n",
      "Conv/weights:0\n",
      "INFO:tensorflow:Summary name Conv/weights:0 is illegal; using Conv/weights_0 instead.\n",
      "Conv/biases:0\n",
      "INFO:tensorflow:Summary name Conv/biases:0 is illegal; using Conv/biases_0 instead.\n",
      "Conv_1/weights:0\n",
      "INFO:tensorflow:Summary name Conv_1/weights:0 is illegal; using Conv_1/weights_0 instead.\n",
      "Conv_1/biases:0\n",
      "INFO:tensorflow:Summary name Conv_1/biases:0 is illegal; using Conv_1/biases_0 instead.\n",
      "fully_connected/weights:0\n",
      "INFO:tensorflow:Summary name fully_connected/weights:0 is illegal; using fully_connected/weights_0 instead.\n",
      "fully_connected/biases:0\n",
      "INFO:tensorflow:Summary name fully_connected/biases:0 is illegal; using fully_connected/biases_0 instead.\n",
      "fully_connected_1/weights:0\n",
      "INFO:tensorflow:Summary name fully_connected_1/weights:0 is illegal; using fully_connected_1/weights_0 instead.\n",
      "fully_connected_1/biases:0\n",
      "INFO:tensorflow:Summary name fully_connected_1/biases:0 is illegal; using fully_connected_1/biases_0 instead.\n",
      "fully_connected_2/weights:0\n",
      "INFO:tensorflow:Summary name fully_connected_2/weights:0 is illegal; using fully_connected_2/weights_0 instead.\n",
      "fully_connected_2/biases:0\n",
      "INFO:tensorflow:Summary name fully_connected_2/biases:0 is illegal; using fully_connected_2/biases_0 instead.\n"
     ]
    }
   ],
   "source": [
    "cross_entropy_loss = tf.reduce_mean(\n",
    "    tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=y))\n",
    "\n",
    "l2_loss = tf.add_n([\n",
    "    tf.nn.l2_loss(w)\n",
    "    for w in tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES)\n",
    "])\n",
    "\n",
    "for w in tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES):\n",
    "    print(w.name)\n",
    "    tf.summary.histogram(w.name, w)\n",
    "    \n",
    "total_loss = cross_entropy_loss + 7e-5 * l2_loss\n",
    "tf.summary.scalar('cross_entropy_loss', cross_entropy_loss)\n",
    "tf.summary.scalar('l2_loss', l2_loss)\n",
    "tf.summary.scalar('total_loss', total_loss)\n",
    "\n",
    "optimizer = tf.train.GradientDescentOptimizer(\n",
    "    learning_rate=0.3).minimize(total_loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:25:56.449132Z",
     "start_time": "2018-06-01T06:25:56.438340Z"
    }
   },
   "source": [
    "需要注意的是，上面的网络，最后输出的是未经softmax的原始logits，而不是概率分布，\n",
    "要想看到概率分布，还需要做一下softmax。\n",
    "\n",
    "将输出的结果与正确结果进行对比，即可得到我们的网络输出结果的准确率。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:39:50.010829Z",
     "start_time": "2018-06-01T06:39:49.997501Z"
    }
   },
   "outputs": [],
   "source": [
    "pred = tf.nn.softmax(logits)\n",
    "correct_pred = tf.equal(tf.argmax(y, 1), tf.argmax(logits, 1))\n",
    "accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "saver用于保存或恢复训练的模型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:52.127795Z",
     "start_time": "2018-06-01T06:32:52.103115Z"
    }
   },
   "outputs": [],
   "source": [
    "batch_size = 100\n",
    "trainig_step = 1100\n",
    "\n",
    "saver = tf.train.Saver()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上定义的所有操作，均为计算图，也就是仅仅是定义了网络的结构，实际需要运行的话，还需要创建一个session，并将数据填入网络中。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:35:22.270272Z",
     "start_time": "2018-06-01T06:33:18.829198Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after 100 training steps, the loss is 0.36547, the validation accuracy is 0.9142\n",
      "after 200 training steps, the loss is 0.221446, the validation accuracy is 0.952\n",
      "after 300 training steps, the loss is 0.0655644, the validation accuracy is 0.9676\n",
      "after 400 training steps, the loss is 0.184705, the validation accuracy is 0.969\n",
      "after 500 training steps, the loss is 0.0575169, the validation accuracy is 0.9772\n",
      "after 600 training steps, the loss is 0.120787, the validation accuracy is 0.9744\n",
      "WARNING:tensorflow:From D:\\Anaconda3\\lib\\site-packages\\tensorflow\\python\\training\\saver.py:966: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use standard file APIs to delete files with this prefix.\n",
      "after 700 training steps, the loss is 0.223587, the validation accuracy is 0.9772\n",
      "after 800 training steps, the loss is 0.171369, the validation accuracy is 0.9776\n",
      "after 900 training steps, the loss is 0.0414988, the validation accuracy is 0.981\n",
      "after 1000 training steps, the loss is 0.140962, the validation accuracy is 0.976\n",
      "the training is finish!\n",
      "the test accuarcy is: 0.9815\n"
     ]
    }
   ],
   "source": [
    "merged = tf.summary.merge_all()\n",
    "with tf.Session() as sess:\n",
    "\n",
    "    writer = tf.summary.FileWriter(\"logs/\", sess.graph)\n",
    "\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "\n",
    "    #定义验证集与测试集\n",
    "    validate_data = {\n",
    "        x: mnist.validation.images,\n",
    "        y: mnist.validation.labels,\n",
    "        keep_prob: 1.0\n",
    "    }\n",
    "    test_data = {x: mnist.test.images, y: mnist.test.labels, keep_prob: 1.0}\n",
    "\n",
    "    for i in range(trainig_step):\n",
    "        xs, ys = mnist.train.next_batch(batch_size)\n",
    "        _, loss, rs = sess.run(\n",
    "            [optimizer, cross_entropy_loss, merged],\n",
    "            feed_dict={\n",
    "                x: xs,\n",
    "                y: ys,\n",
    "                keep_prob: 0.6\n",
    "            })\n",
    "        writer.add_summary(rs, i)\n",
    "\n",
    "        #每100次训练打印一次损失值与验证准确率\n",
    "        if i > 0 and i % 100 == 0:\n",
    "            validate_accuracy = sess.run(accuracy, feed_dict=validate_data)\n",
    "            print(\n",
    "                \"after %d training steps, the loss is %g, the validation accuracy is %g\"\n",
    "                % (i, loss, validate_accuracy))\n",
    "            saver.save(sess, './model.ckpt', global_step=i)\n",
    "\n",
    "    print(\"the training is finish!\")\n",
    "    #最终的测试准确率\n",
    "    acc = sess.run(accuracy, feed_dict=test_data)\n",
    "    print(\"the test accuarcy is:\", acc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:58:16.766415Z",
     "start_time": "2018-06-01T06:58:15.918700Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from ./model.ckpt-1000\n",
      "1.0\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAd0AAAHiCAYAAACtERYWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3XeYFMXWBvD3bGAJS855yVlUEAMqoCAmFAMmBDFeL2IWrwEFc7ifOaMiKgZUxHjFAKIgkpNkFBDJOcMCu+f7o3u7u4bpZXZCzy77/p5nnz09VVPdPTXdNV3VQVQVRERElHgpyV4AIiKi4oKNLhERUUDY6BIREQWEjS4REVFA2OgSEREFhI0uERFRQNjoEhERBaTINboiMkREDojILhEpE+F7/hKR/SIyIp88KiK7ReSx+C1t8ERknIjsE5GJyV6WSLFO81fU6pT1mb+iVp8AICLD7fpZEWH+pnb954jIdT55OotIrp3vzLgucIBEJMNehwMi8ujh8iel0bUX0PuXIyIvFaCIkaqaqaq77fK+Cylvv4j8kZdZVRsBeDyCctuq6v2e5RwqIovtL0a/MOtxu4isE5HtIjJMRDI8aVki8rOI7BGRRSLS1W+mdqUNE5Eddnl3eNLqishkEdkiIs+EvG+MiLT3vqaqpwG4MYJ1jRt7+d8Wkb9FZKeIzBKRswpYTGidDhSReXZ5y0VkoDcz6zSxRGSAiEwXkWwRGR5FEaH12cX+7LaH23GzPhNPRCqJyGixfrj8LSJXFLCIp1U1y1Oe72eiqktUNRPAhMOUucb+noyxy6wpIl+JyBqxfmRleTPnN087/XS7LvfYdVvfb8b51b9dznIRWSsil3peryAiM0WkrGdds+11/eAw6wogSY2u/SFn2gtaHcBeAJ/GUN5ZIWVOiqU8jzkA+gOYGZogIt0B3APgdABZABoCeMiT5SMAswBUBnA/gM9EpKrPfIYAaAKgPoAuAO4W95ffvQDeBdAAQM+8Ddj+IixT1enRr17cpAH4B0AnAOUBPADgk9ANpoAEQF8AFQGcCWCAiFwW22ICYJ1Gag2ARwEMi1N5u+2yBh4uYwGxPiP3CoD9sPa5vQG8JiKtYihvCPw/k2jlAhgD4KKCzlNEqgD4HNb+pxKA6QBG5jOv/Or/eQA9YO17XhORVPv1JwA8qao7o1k5AICqJvUPwFUAlgGQCPMPATAin/QsADkAGhTwfQqgsU/aRAD9Ql77EMDjnunTAayz46YAsgGU9aRPAHCjT/mrAZzhmX4EwMd2/B2AZnb8MYBLAJSzvywVfMrrB2Bikut1LoCL4lGndp4XAbzEOg28Hh8FMLyA7/GtFwBdAayI5nvA+oypHsvAanCbel57H1YDEsn7hwN4NNLPxPPaeADX+ZTZGcAqn7Q0u76zClAPNwCYFLLOewE0D1N+vvUP68dS3uvrAFQD0AHAmIJ8RuH+CsOY7lUA3lN7qQFARLaJyMlRltcXwARVXR6XpfPXCtav7DxzAFQXkcp22jI1fw3NsV83iEhFALXClJWXdx6AbiJSAUB7AAtgfdGeV9VtcVqXuBKR6rC+1PM9r0VdpyIiAE7xlpcgrNMIxbiNBoX16WoKIEdVl3hec9ZBROrZdVovksIi+EziLoJ5GvWt1tDGXz7LdLj63yAibUWkLayj762wjn5viXU9ktro2hXcCVbXjENVK6hqtCcZ9IX1iyPRMgFs90znxWXDpOWll8WhMkPeH5r3CVgNzi+wuofSARwF4GsR+VBEfhWRAdGuRLyJSDqssY13VXVR3usx1ukQWN/Vd2JfwnyxTiMUY30GhfXpynd9VXWlXacrC1BeXhmHlJcgh5tnQes0v7w3AngBwFAAfQD8G8BYACVF5Ht7LLhTNCuRFs2b4qgvrC6WuByV2r+8awD47DD5voO1kQDAv1Q1ogHwELtgdSHlyYt3hknLSw83DrDLk74vNK+qbgFwqb3cKQB+hfWFuAfWL+x+AGaKyDhVXRDFesSNvXzvw+rGistOxt5Z9QVwiqpm55OPdXoEYX3GXUHWN9Ly8so45DMJR0R2eSZbJmCeBa1T37yqOhtW9zdEpCaAZwCcCOuH1W2wznn4VUTqe3tpI5Hs7uW+CDnKjdFVAD5X1V35ZVLzxKtoNmbA6ups65luC2C9qm620xp6z3Cz0w/pHlXVrQDWhikrXFfqDQAmq+o8AG0ATFfV/QD+ANA6yvWIC7sL+G1YJ2lcpKoH4lDmNbBPhFHVVfnlZZ0eWVifcbcEQJqINPG85rcOh1XAzyTvPZmev0iPqAsyT6O+xbpcrZHPMkVc/wCeAzBIVffCrdMVsHo0/E6885W0RldETgJQG/E5yxgiUgpAL8Sxa1lESohISVhn0qaLSEn7lywAvAfgWhFpaY81DMqbtz1uMhvAYPs9F8DqbhrlM6v3AAwSkYoi0hzA9aHrISLVANwEq6sVAJYD6CIimbDGkZbFYZVj8RqAFgB62F/OmIhIb1iXkHRT1bitG+s0MiKSZn9OqQBS7XWOumdMRFLs8tKtSSkpIiXisJyszwjY45ufA3hYRMqISEcA58PqmYrWYT+TaNj1mXdpV4Y9Hck8RwNoLSIX2e95EMBc7zBXnkjrX0S6ASipqt/YLy0HcJpYZ31nANhc4BU83JlWifoD8AaA933SdsHqTgyXNgRhznAEcDmAv+FzFrTf+zzph5wZCevMOw356+xJvwPAegA7YI03ZnjSsuz37wWwGEBXT1pvAPM90xmwLqfYYZd3R5jlew9AL890XQBTYA3wPxOStx+CPTOyvv3Z7LPrLu+vd7R1CuvLfSCkvNdZp4HV6ZAwn9OQGOqzc5jyxrM+g6lPe56VAHwB6/KtlQCu8KTVs+u0ns97h+PQs5cj+UzGo4BnL4epT410nrDOjl9k1+l4eM5+BvA6PPuQ/OrfM6/ZAOp7XjsdwApYR9yXHe4zCrveQVZ6nL44g+wvzTYAZSJ8z2L7CzUsnzz7YA2kP5LsdYzx8/kR1rjE2GQvC+u0eNYp6/PIqk97md+06+evCPM3set/D0Iu5fLkOdVu8LYB6J7sdYzhs8mw12E3gMGHyy/2m4iIiCjBkn0iFRERUbHBRpeIiCggbHSJiIgCEujNMbql9OIAcpL8mPupxLtM1mfyJKI+AdZpMnEbPbL41SePdImIiALCRpeIiCggbHSJiIgCwkaXiIgoIGx0iYiIAsJGl4iIKCBsdImIiALCRpeIiCgggd4cgygRVjx6ohPnlDTvBVC11UYn/r2t36NSgUbjrnbislNLGWnVX5wU6yISEQHgkS4REVFg2OgSEREFhI0uERFRQDimS0XS1m+bOPG8o1+O6D0H8rn1+6IubznxB+1rGmmf/NjJiXMWLo1wCakwkXatjOlvv3rfidu8PsCJ6z7C8fugpVYo78SLX27oxN5tEgAGbWjnxH/0bmqk5SxYkqCliz8e6RIREQWEjS4REVFA2L1MRYK3OxkAfjv644je9/o2t7vq2d+7OXFW/Y1Gvh9afu7EvcuuNdIe61fFiRv+h93LRdGG48oZ0weR48Sl1/CRs8mU26COE//R+Q0nDh0OerTaDCdue8FJRlpddi8TERFRKDa6REREAWH3MhVaB093z1Yc1/aVkNR0J3p+q3sm48+XtjezrdnghE23TnfilJIljWyPT2njxPdV+cNcjooHI15mKpy2HpVjTK86mO3Eld/+PejFKdbS6tYxphsM/TNJS5IcPNIlIiIKCBtdIiKigLDRJSIiCkiRHtPdfP2JxnS9Pu7YwKIN1Y20/dnuGGDtj9y49KpdRr7c2QviuYgUg121SzhxSsjvQ+847vjz3PHYnGWLIyr7z4eOMaY/rPSMZyrDSKszhr9NiyLteLQTTzj3WSOt0683O3FjzApsmYqrlQ+6l/i0O9Pcxz5dc0KBy8s8ybzk758H3PKrzHXPwSj15dQCl51o3JsQEREFhI0uERFRQIp09/LdAz80pi8qs9WdaJTPGzu74YqDe4ykFzZ2iX3BIjR1Q30nLvNMeSMtbeyM0OzFToX33Es5Lp5+pZEmW3c48cG1Kwpc9nVn/2RMZ6Zk+OSkompLy1JOXDO1tJFW+7P00OyUQHP/9ZITH9CcfHJGZnzbD8wX2rrh6N3uA0uG7expZEsbl/z9Ko90iYiIAsJGl4iIKCBsdImIiAJSpMd0X7zvMmP6waPc3xAVF5qPqNjaQpy4xFHbnPjp1p8b+Z6rOcWJv92T6cTnlDYvLcrPXt3vxFOyyzhx55IHzIyeeTW+9F9GUtOxEc+uWIjHQ6pXPOZeYnZthf8LSXVvC3nn2hOMlLI/LXSXI+aloKCc3t89J+CL3RWMtMzx7qVlrNPESB/vjq2mS2rM5c3an+vEKw5UNdIuKLPFiS/JdG/9esn7Q41859Zuh2TjkS4REVFA2OgSEREFpEh3L5f5bErItH/ecj6vv1SjszH9aMcs9z2/uHe4erpz44iXK22v2w1SZq77QPTKv44y8rUp4bkz1gpewpAI2/q4Xcq/9XW7lMunmE8Z+j3b7f6a/ah5t6pSOwrfXW3oUKmtmhnTj1f7yInf3mE+2SZn2/ZAlqk42duzgzF9dc1Pndh7mVCklwy1HnujMV11rHtZX8Z2s4x7O7vHj3/0etG3zFX3uneuqvPEpIiWI954pEtERBQQNrpEREQBKdLdy/FwcN16Y7rMKHfa24FR5rPNUZW//jq3e7NVCfPj/r8tbndY1jvLzOWKam4UatOx7lnsoV3KXleNv86Jm37B7uSiaHW3yr5pM3bWD3llb2IXppjwduk/+qx5pnD7Evu9OX3L8N5BatDPFzlxi7sXGflyduyAn2ZL3QegTD3P3c47ZOwz8n3376ed+IySdxtpWY+7d6vS7GzfecWKR7pEREQBYaNLREQUEDa6REREASn2Y7qJkFa/rhO/fN/LThx6V5ZPX+jqxJXX/g6K3f4fzbG735t7H07vjvW0/f0qI1+LO/9yYt6hqGja0fKAb9rsl482piuA21s85HrOUzHHcP1d8/eZxvTOS92nQTVd5Z5PUZDt0HvHuv7D3UuNpv/reSNfzVR3XjOvNdMu+tzdJ+ichUgUHukSEREFhI0uERFRQNi9nACLbq/txMdluA9amL/fvEyh0oI9gS3TkSytYZYTP9L4UyOtoucyoRmeqwDqP2J2XuVs3ZqQZaPEyj7rOCf+8oyXjLSHN7k3t680aq6RlgsK0n3r2zvxjuvMS7tyVi2N67yyRm1y4gd6mg8vebLGtLjOKxo80iUiIgoIG10iIqKAsHs5DrLPOc6Ynnnxc54p9ybd/771ViNfqUm881E8NPpktRMfU8L/d+TlnhuoN52T/G4mit2q09xd2FElzDuOXbWijRNX223e3YjiL79n5s491vt88/h2Jx9C3CG9tBRzICG/ZVzzkBvX6Bn3pXLwSJeIiCggbHSJiIgCwkaXiIgoIBzTjYOVZ5m/XTLFHce9fHk3Jy49Zo6RT0HR2nqV+/Smh6p77zqVYeS7aoV7168Wd//pxLzr1JGhausNTpyj5vhd2pcVg16cYmfxv0s7caQPp0+0FRe6lyR9VtU8b+aApnpic3lrDXbjRF5SxiNdIiKigLDRJSIiCgi7l6OUUrasE/c5ZaKRtiPXfXDyhscbOnFGNi9TiVZa7VrG9Cm3THHizJSM0OyO3xc0duKmW/n5HwnSGrgPtfi/Zu4dyN7cXtfIV2kYH2qQaINO+Top802rW8eY3tnO3T+8fvWrEZUxNdu8xEz2H4x9wSLAI10iIqKAsNElIiIKCBtdIiKigHBMN0pLh7Ry4m+qmGMI5y+9yIkz/sdxxHhYeJ85XvdFjfBjSV3+6GVM8zKhI8/Sf7njdyd4hvOvn9nFyFcX84JaJArYgodqGNPzz3g5oveN2lXFiV+7y9xXlFwYzG15eaRLREQUEDa6REREAWH3coS2X2k+DHnupS868V8HDxhpu55yT2fPwNrELlgxMeO850JeCX+ZUPn+5r1kDvLh9Eec3Lr7wr6+d1vJsK/TkSF9fE0nfqLmqKjKGL76JCcu+XVynvLGI10iIqKAsNElIiIKCLuX8+G9C9JtD4w00jLE/egum9PHSKv6Hc9YTpYD1csb0+n7axe4jJyNm4xpzc52Yslwu7VTq1aBn5yqFYzppXeWiGjemuM+gLv5zX8aaTk7dkRUxpHu1eNHhH299nf+DyinxEgVdzgnvwfE77jiBN+0hx5+24m7lAo/dBBa/qEPV4is7vW01RHlSyQe6RIREQWEjS4REVFA2OgSEREFhGO6ISTN/UjafrPKiXtlbjbyfbCzmhNXf8D87ZLIByBT/r79bFjMZZw063JjetP6ck5csepOJ57S7sOY55WfloMGGNMN7y6eT83Z16ODMX1ySe+lHtyFJdOTIy924kuufd4336//fcWJ83vY/QGNbL75leHVeuyNxnQTzIxsBgnEI10iIqKAsNElIiIKCPtmQrVt5oSPVHvfN9srj7s3y64wp3h2+wXp/AW9jemxrT9L2LwmHfNRVO/bo/ud+ID6DzKcPbefE2+f7X/ZUe2JwTxUu7BbeZ7Z5+i9XO/hTW2cOPPLGUa+CHsqKQYNR7qX10290rwjWIcM/8t/YhX6APqh6zo58db+7sMQmi8PuewuYUsUOR7pEhERBYSNLhERUUDY6BIREQWk2I/pprZsakzf8PGXYfO1HHaTMZ31/uSELRMdqlT35cZ0q8fdy2k0wm9x2eZbnLggl/u0mnC1O6+VZXzzNfxslzsx9Q/ffBWxNGxMrtRy7mVa/+n4P998H353qhM3PMhzK4KWs2CJEz94x3VG2j893PMalpz1Rlzn23+YeSlQ3ccmeaYK95PFeKRLREQUEDa6REREASn23cuL+lc0pnuUDv8klzrj95svKC9ISKYG98XWlXgu2kU+L8yNaV5UcLmeJzst2FPLSOu6ur0TN3l8vhMXhstBirNSX5oPhW/qGak79XJ3eC6933oj35hW7hPczph3mRPnDq9m5FP3AVzImr3RSCtKdc8jXSIiooCw0SUiIgpIsexe9t5AfWyPZ0JSSwe7MER0CPV0Ly9ub6aVwN9OXJS6FYuzch95rvYIueHbBXD3x2WwzJOyDH6Kcr3zSJeIiCggbHSJiIgCwkaXiIgoIMVyTHdNx1QnrpfmP4brfVB9+g7zkiFeMERERAXFI10iIqKAsNElIiIKSLHsXs7PE5tbOvHv3bOcWNf638CeiIgoEjzSJSIiCggbXSIiooCw0SUiIgpIsRzTbXiP+4Sas+85Np+c6xK/MEREVGzwSJeIiCggbHSJiIgCIsqHsRMREQWCR7pEREQBYaNLREQUEDa6REREAWGjS0REFJAi1+iKyHAR2S8iKyLM31REdolIjohc55Ons4jk2vnOjOsCB0hEMux1OCAijyZ7eSIlIkPsZd4lImUifM9f9vdgRD55VER2i8hj8Vva4InIOBHZJyITk70s0Sju26yIdLWXM1dEuiZ7eWLF7TWy9fGT1EZXRJrYO5OCLvjTqpoVprxKIrLRu3NS1SWqmglgwmHKXKOqmao6xi6rpoh8JSJr7C+DMT+7gRsmIjtEZJ2I3BGSfrqILBKRPSLys4jU95uxiGTZefbY7+kaUs5yEVkrIpd6Xq8gIjNFpKxnXbPtdf3gMOsadyIyQESmi0i2iAyPooiR9ue/2y6vi/2ZbA+3s1bVRgAej6Dctqp6v2c5h4rIYnsH2C/Metxu1+d2u34zPGm+9RSmHN/vh4jUFZHJIrJFRJ4Jed8YEWkfsq6nAbgxgnVNuEK+zYqI3C8iK+3P/WMRKeeZ1yUiMsmuv/H5FRxBWQNFZJOIzBOR1p7XO4rIF96yVPUne31WHmZ9AiMiLcT6MbddRP4UkQsKWETo9lpBRN4VkQ323xBv5hi21x72Z7zLrruWnrQMEXnO3kdvFZFXRSQ9n3U+zd5n7hCRZSJygyetrYjMt+v0ds/r6SIyRUTqRrk+h0j2ke4rAKbFsbynACyMU1m5AMYAuMgnfQiAJgDqA+gC4G6xf3GLSBUAnwN4AEAlANMBjMxnXh8BmAWgMoD7AXwmIlXttOcB9ABwJoDXRCTVfv0JAE+q6s5oVi4B1gB4FMCwOJW32y5rYJzKyzMHQH8AM0MTRKQ7gHsAnA4gC0BDAA95suRXT6GGwOf7AeBeAO8CaACgZ14ja/+oWqaq06NfvYQrzNtsXwB9AHQEUAtAKQAvedK3wNqenoylLBGpCeBaWN+P1/PKE5E0AM8AuC32VUkcezm/BPANrP3TDQBGiEjTGIp9DkBpWNtNBwB9ROTqGJezCawDiBsBVADwNYCv7OUHrG21PYDWAJoCOBbAIJ+y0gGMBvAGgPIALgXwrIi0tbM8AeAuAG0BDBKRGvbrdwAYpar/xLIuXklrdEXkMgDbAIyNU3knwvrw34lHeaq6XlVfhf8Opi+AR1R1q6ouBPAmgH522oUA5qvqp6q6D9YOuK2INA+z3HlflsGquldVRwH4A25jX0ZV56nqHAD7AVQWkQ4AGqjqJ/FY13hQ1c9V9QsAm+NU3lRVfR/AsniU5yn3FVUdC2BfmOSrALytqvNVdSuAR2DXaQT1FCq/70cDAONUdTus71dD+yjqHgD3xWE1E6Kwb7Owfpy+rar/qOouWA36pSJSGnCOOD+B9QMxlrLqAZilqjsA/ASr8QWsxvYrVV0Rp/VJlOawfkg8p6o5qjoOwG+wfmREqwes3ow99vq/DeCaGJezO4AJqjpRVQ/CqoPaADp55vmiqm5R1Y0AXsxnnpUAlAPwvlqmwfqxl3fknLdNrgawFEA9EakHa/t+Lsb1MCSl0bV3MA8DuDNMWj0R2WavcKTlpcL6BT4AQMLv9iEiFWF9aed4Xp4DoJUdt/Km2V0wf3nSvVrBOrrxHrF6y9pgd320hXX0vRXWr/Vb4rAqgbHr9ORkL8dhGPVmx9VFpDIOX0+OCL4f8wB0E5EKsH6pL4DVwD+vqtvitC5xVUS2WbH/vNMZsHoc4lnWnwDa2PXXFcB8u/vxMgD/F8W8giY+r3m7yaPZXkM/r9Z+GQtQXmiZ3nLDpdcRkfKhBanqelg9VVeLSKr9g68+gLxhjXkAzhCROrCO1v+C1YjfraoHYlwPQ7KOdB+B/SsyNEFVV6pqBVUtyPjHLQCmqOqMuC1h/jLt/9s9r20HUNaTvh0mb3poWfnlvRHACwCGwvol+m9YRxolReR7scYYO6GQs+u0sJ8IFFoXeXHZMGl56X516n1/aN4nAJwC4BdYDU86gKMAfC0iH4rIryIyINqVSJCisM1+B+A6scbeywP4j/166XiWpaqbATwGYByAc2B1S75g57lARH4RkS/tHXhhtAjABgAD7THLM2AdPTqfUxTb6xgA94hIWRFpDOuIM5rP3etHAJ3EOmmuBKxeoBKecr8DcKuIVLW7g/MORPzm+xGABwFkwzpf4H7P9/kuWPvWrwDcDmtYYSeAZXZd/iIivWJcHwBJeMqQiBwN69fhMXEqrxasD7tdAd6zyzPZ0jejv7z3l4PbTVkOViXlpZcLeY83PbQs37yqOhtAZ3u5a8IaMzoR1g77NlhdZb+KSH0txvf0FJHvYDVkAPAvVY3mZLLQusiLd4ZJy0v3q9O89EO+H6q6BdaYEkQkBcCvsH5c3QPrF3c/ADNFZJyqLohiPeKqCG2zwwDUBTAe1r7tGVhdkKuiWMx8y1LVj2DtxCEi58Dakc+C26NxHqyj3suimHdCqeoBEekJa4z6P7DOOfkE1jpE6xa7vKWwhpg+AnC5X+ZItldVXSQiVwF4GUBNACNg9Qrl1edjsMZ6Z9vL/ias7+iGMPNrDuu8mgtgNeZNAHwjImtU9VtV/RvA2Xbe0gAmwerefsl+37cA5onIWHv7jVoyjnQ7wzp8Xyki62D9wrhIRA45sSVCHWBVyAK7vBcAdBDrjNHUcG+wz7rL+yvwGYX2eN9aWIPuedoCmG/H871pYp1W38iT7jUf1pie94iprU/e5wAMUtW9ANoAmG6Pn6QD8Duhp1hQ1bM8dRrt2dtGvdnxevvIJuJ6iuD74XUDgMmqOg9une6HNV4ca/dcvHRGEdhmVTVXVQerapaq1oH1ea+2/wok0rJEpBSss1jvhLUj/8ce650Gq/eiUFLVuaraSVUrq2p3WOPSU2Mob4uq9lbVGqraClbb4ltepNurqn6mqq1VtTKAwbC6hKfZaXtVdYCq1lbVhrAa+xmqmhOmqNYAFqvq93bdLobVkJ4VJu+DAN6yu6TztsntsBr7xhF8HPlKRqM7FFYDdLT99zqsle8eZXnfwdoh5JX3IKxfnEf7fPgRE5GSsMZxACDDns7zHqyz3Crav6KuBzDcThsNoLWIXGS/50EAc1V1Ueg8VHUJrF9qg0WkpFin7h8FYFTIsnQDUFJVv7FfWg7gNBFpZS9jXE5gipaIpNnrmgog1V6XqHtSRCTFLi/dmpSSdhdTrMtZwi5XAKTb5eZtB+8BuFZEWtrjsoNg12mk9eSR3/cjb1mqAbgJ1ol2gFWnXUQkE9ZYb1xPIotBkdhmxbr8qJFYWgJ4FsDDqpprp6fadZ8GIMWux7CXmByuLI9BAIar6hpYlwQ1E5HqsM5YLyz1dwgROcpe/9IichesH0HDYyivkYhUtj/js2D9mIz5XgEi0s4usyqsM4+/ztuPikhtEall19EJsK4WGexT1CwATcS6bEhEpBGAc2GedwG7rjsDeM1+KW8/Wx3Wj6rYL/tS1aT+wdrhjPBM14PVPVfPJ/9wAI/mU14/ABPDvD4ewHU+7+kMYFWY1zX0z5OWAasLageA9QDuCHlvV1hjJ3vteWd50l4H8LpnOsvOsxfAYgBdQ8rKgLXDr+957XQAK2AdUV1WkM8ogfUY+nkN8aTvAnBKJN8BT52Eljf+cO8LU3+Nw3wPQsvt7Em/w67PHbDOqs2IpJ4A9IZ1xnpE3w87z3sAenmm6wKYAutkuWci+V4n4y/0c0ch2WZhXTayGMAeAH+Hfub2fELrfni47+jhyrLzNIN11JXmeW0ggE2wukHbhORfgZBtO4l1+F/7e7bP9BwTAAAgAElEQVQL1o+g0O2koNvrJbCGuvbA2ld1j+R9IenhtteJsIZltsBqdMt40k61P9M9dl31DnnvdwDuC1nGeXZ5q2CdDZ0S8p6fARzvmW5r1+WmMN+nfNfHdz2TXflRfFnetL8Qf0WYvwmsyxz2AOjnk+dUWDvSbeG+LEXlD9aOfhusa1wHJ3t5CrDcg+xl3ubdqA7znsX292BYPnn2wTqB6ZFkr2OMn8+P9o5ibLKXJcrlL9bbLKwfyNvs5e2S7OWJw/oU++01kvXx++PzdImIiAKS7DtSERERFRtsdImIiAIS6HW63VJ6sS87SX7M/TTcXWhiwvpMnkTUJ8A6TSZuo0cWv/rkkS4REVFA2OgSEREFhI0uERFRQNjoEhERBYSNLhERUUDY6BIREQWEjS4REVFA2OgSEREFhI0uERFRQNjoEhERBYSNLhERUUDY6BIREQWEjS4REVFA2OgSEREFhI0uERFRQNjoEhERBSTQh9gXFjldjnXiAUM/MdJea9I4YfPdeekJxnSF2ZvcZVr8Z8LmSwWzre+JxvSUJ19z4pav9Hfiek9NNfLpwYOJXbAjXFr9uk5cbeQ2J/5lRksjX/NX3bSc+YsTv2C21KpVjenNZ7n7ioojZzqxZmcHtkxU9PBIl4iIKCBsdImIiAJSLLuX/+6e4cSVUncFNt915+w3pg/0cX/zVDo3sMWgMNJq13LiRx58yzffgptedeKzXjzFSNOdO+O/YEewtBrVjemHx49y4mbpuU582uYaRr6c+UsTu2Ae3i7l3hNnGmknlBztxDf98S83Ydb8hC9XUZZapbIxvfi5ek7cuYlbt6s7HTDyHSnd9jzSJSIiCggbXSIiooCw0SUiIgpIsRnTlfQSTnzaabOTsgxlZ5U0pi+59hcn/rlCHSMtZ9v2QJaJLBu613fiM0of8M137PRLnbjqriUJXaYjUVqd2k5cfuQeI+2oEqlO3OynG524yVXmWGqQFj6a5cSXZI4x0o59/m4nrjVrUlCLVCRtGHCSEw++9T0j7ZzSP4R9T88qPYzpg6vXxH/BkoBHukRERAFho0tERBSQYtO9vPMC9y5UL9Z+yYlbfDHAyNcEUxK2DNkV1Zi+peIiJx5ftoWZmd3LCZVSurQx3f2WiRG9L+Pjiu6Eqn9GCmtrR/euU19kveKbr8WgDU4c5H2+9MS2xvSf577hxJ3+6GWk1R3mbr85iV2sIim1aSMnfuvO55346BJms5OL8Na+VtaYrvkv99Kxg2vXxb6AScIjXSIiooCw0SUiIgoIG10iIqKAHLFjutrxaGP6ladecOIRO9zLQ5oPMi/7SOTYzIlnzEtg6VQQ2SeZY+iPVnvbN++eXPf2neU+nJywZToSeZ8cBAAbz9/nm7f9/93sxDX+Ce4SHO847qAP3vXNt+tb83aUZTYvS9gyHQkW3uOe/+C9HCxSU9p9aEwv+d3dDi98/w4jreFjs5w4d5//d6ww4JEuERFRQNjoEhERBeSI7V7eeq95t5s6ae6FB3fcfI4Tp2+dkdDlSKvpdkm9U8+8o80B5W+eZFl+YeTdXRcv7emZOjLuihOUf17INKaXdhjuxIM2mENAtd9xn84T5CU4qzuXceKOGeYFLK0nXeXE9V7iXafyk9qyqTH90+nPe6ZKOdFTm82hnenb3KcMjWxk7iO9mnruKvhm79eMtKeGne/Eucv/jmh5k4V7fSIiooCw0SUiIgrIEdW9vPn6E5340zb/NdLe236UE6f/lNguZa8FD7tnbx5Qs9PsqhVdnThnw8bAlomAc46b45u2PXevMX1giPuw9RR2LxeIqhjT3m1gyuYsIy117wYkSkpZ8+5Gix9r6cRfnPesE+ci3chXr9cfCVumI82mDubD6bPS3Lu+3fDPqU686oRdRr6UMu5QYLsb3TPY77r+EyNf77Lu9+NU89kx+HrUSidecE7hvnMVj3SJiIgCwkaXiIgoIGx0iYiIAnJEjemm9NzkxLXSMoy0tz8804nrILGn/qe2aubEI053n1KSrebD0Vc+655iXyY7cU83Ikv22cc58cu13/TNtyrksTYpv8wKn5Fi8r/mXxjT147v4sQrd9Z04v1vm3eCitS6U9ynQJ19/Gwj7atar3qm3HHcjrMvM/JVxNKo5l0c5Zi7XOTC/fznvtHGiSvhdzPf7t1OXPMZd9/8SY/jjHyXl/3GnVDz0q712e6Yve7Ljnyhk4BHukRERAFho0tERBSQIt29nFq1qjE9qOm3vnnrPB7c3WQW9a/gxO0z3EskXtna0shXZhS7lIO0/rj0w2cC0OOb24zpJmA9RavaS6WM6Z+Hutd6dCll3pj+7Xo/O3EK3EuNcp9VRMMoA/5lfLTTvSSs8n2RPWCdDlX2orW+adu7u13Ild6JrLwH638V8or/MeKEWc2duOnWqZHNIEl4pEtERBQQNrpEREQBKdLdy1LavC1J99LbnbjDtL5GWg0sDGSZAKBK1pawr3+wvL2ZD0vC5qPEKHHMVt+0hfvdu+I0f3GTkRbkzfePNGnjzLu/vXDyaU78yElZRtqqM9wu4D97vO7EU7PNu1pd+cONEc27yXvuWazffjrMN9/TC7o7ce05833zUf52jqppvtDKDfu1dIdofj2ug5Ft4zHuQzH0XHff2Trd7CZeeMC9+qOV5+EHADD6rJec+D8nXO8mTJ57+AUPGI90iYiIAsJGl4iIKCBsdImIiAJSpMd0c7dsM6Yf2XisE1/RaLqR9mvNRk4c7ydPpNWva0z/dvTHnin3d83eyVVC3skx3UTbd647fjT9OO+Dr82H2C8+UM2Jc5b8lejFKrYOrlvvxKU/X2+kNf3cjc++8Vj4aYrILglJOcq9jMR7+RAAPLqptRPXv9U9FyTkZmRUADW+Wm5ML7l3vxMPrLzAif/zhXl+jd/lXJf+dY4xvfcW9xLRCz4ab6RdXe4fJ/7rFnef22jyYRY6CXikS0REFBA2ukRERAEp2t3LO3ca0z+sdruTJhz9oZG29pvybtobJ6KgtrU0u0Ays9wuqRNqrTCXy+c+NhLdjXUoBnuruN3I6ZLqm+/uGRc6cQMUvssMqOBWDnbrO7QL84fH3IeqZ/5TCPsgi6DQYbsbBrp3dnvn/5514qbpZcw3eh5e0PgH93Kf5gMWGdlyd7td1E+O62GkXdvTHTp6qr07TvFWW7OLOndOcJeO+uGRLhERUUDY6BIREQWEjS4REVFAivSYbqiKD7m3hew05HIjbXTr4U781GDzIcqRmJ5tjgfmeH6vtC+xPyS3IJx6L/1hTPMJJomX3XNb2Ne9t30EgDpvRfYEIiq8Nt1gnqsx94RXnHjFwb1GWqmNodssxVvmp+6tH6/GHU685RJz29u3PcOJWwx0L9fL8TzcPlSzexYY06c3cc/J+LHVKCcePNg8rqx9IZKOR7pEREQBYaNLREQUkCOqexlT3e7b8mebSX063+LE25pkoKAqv+nfJb3681bG9Izjh4fNF3qJE8VfatNGxvT040Z4U53ou12tjXzpP5lPw6GiZ0+3Xb5pF8++zpiu9vPMRC8OeXi7mjM/9c8X6RO9QvelO0Z7tmfP7vipo0YZ+V6t2dmJ431nwkjxSJeIiCggbHSJiIgCcmR1L+cjdbzbnVR5fHzL3ruirPnC8eHzacejjWn5bXZ8F4Swvks1Y9rvLlQv/9zNmG6CKWHzUdHxRrv3jem1Oe5ZspWfLx304lCAqr7hPgTj+LOucOIp7cw7E956V5YTN7qT3ctERERHNDa6REREAWGjS0REFJBiM6abUCE3oErx+S3DMdzE21cp/N3AAGBGtnsXohZPrTLS+PDyomnVvSc5cccM8zKgydnuOG4qLxE6suW6FxtVfsat903vm3ciW3iZe5eyHh/2NdJ0xvwELZyJR7pEREQBYaNLREQUEHYvx0PIw+n9HmJPiVfttNW+aV/tOMaJczZuCmJxKMF6Xz7WiUMfVH/t9H5OXB/mw0ZSK1dyJ6pVdsKchUvju4AUuJRfZjlx53cHGmkLrnG7l3c+ZnY9l+vlXvqZyLsH8kiXiIgoIGx0iYiIAsJGl4iIKCAc042D3JL+Y7gbc7IDXJLiSTLcp0adX2uOb77N+zOdWLNZL0e63Bz3mGLDgJOMtHOum+DEXyyr6cSF4SHnFD+Nh/5jTL/fq4YT/9rmMyPtzLbXOHHKxMRd3skjXSIiooCw0SUiIgoIu5fjYMSZrxvTC/e73c2XD7/biethUmDLVKzkuHejGbrwZCPptpNWOPH4fxo7cW0Ec/cZSp6Fp77jxLmnmpcTtfrV7UpsPGS3E0f6EHUqGg7+Y9557pMLOjlxn59GGmmbBu5z4moTE7dMPNIlIiIKCBtdIiKigLB7OQ4eXn6eMb371dpOXG8Uu5QTTQ+6jyvIume3kdbiiT5OLLPLgo4s39/vdhcuuLemkfb7lOZO3PyFNUZao3WLnThn3z5Q8eC949ily84w0r4+5i0nvvaE/m7C5LlxXQYe6RIREQWEjS4REVFA2OgSEREFhGO68XC6eVp6GazyyUiJlvPncmO6Xq8kLQgFouTXU51449dmWmNMduKDIDLtucC8jGzKpFpOvLVZGSeuOBlxxSNdIiKigLDRJSIiCgi7l4mIqNjJ2bTZmB7atKETV8TvCZsvj3SJiIgCwkaXiIgoIGx0iYiIAsJGl4iIKCBsdImIiALCRpeIiCggoqqHz0VEREQx45EuERFRQNjoEhERBYSNLhERUUCKfKMrIsNFZL+IrIgwf1MR2SUiOSJynU+eziKSa+c7M64LHGeRrE9hJyJDROSAvR5lDv8OQET+sut9RD55VER2i8hj8Vva4InIOBHZJyITk70skSju22R+RCTDXocDIvJospcnUtxG81eQbbRQNLoiMt5e4F323+ICFvG0qmZ5ysvb6Hd5/lIBQFWXqGomgAmHKXONqmaq6hi7TBGR+0VkpYjsEJGPRaScZ561ReRLEdkiIqtE5MbDrPPNIrLcLmu6iJzsSbtCRNba6Z09rzcSkUl561LA9UkoEWlhf/G2i8ifInJBAYsYaX/eu+3yKojIuyKywf4b4s2sqo0APB5BuW1V9X7Pcg4VkcX2DrxfmPW4XUTW2esxTEQyPGlZIvKziOwRkUUi0tVvpvbOdZhdv+tE5A5PWl0RmWx/V54Jed8YEWkfsq6nAcj3+xRvIlJJREbbO8S/ReSKAhYRuk36fh4xbJM1ReQrEVlj77yzvJnzm6edfrpdj3vseq3vN+P86t4uZ7m9zV7qeb2CiMwUkbKedc221/WDw6xrQojIZSKy0K7Xv0TklAK83dhG7fKOFZFf7X3sehG5NS+N22h4haLRtQ2wKzRTVZvFobynPeVlqmpOjOX1BdAHQEcAtQCUAvCSJ30EgOUAqgM4B8DjItIlXEEicjyAJwFcDKA8gLcBjBaRVBFJs9OOBXAzgJc9b30RwB1xWJe4spf5SwDfAKgE4AYAI0SkaQzFPgegNIAsAB0A9BGRq2NcVACYA6A/gJmhCSLSHcA9AE6359sQwEOeLB8BmAWgMoD7AXwmIlV95jMEQBMA9QF0AXC3uEdo9wJ4F0ADAD3zNmB7h71MVadHv3px8wqA/bC+z70BvCYirWIobwj8P49o5QIYA+Cigs5TRKoA+BzAA7C+s9MBjMxnXvnV/fMAegA4E9bnlPej+AkAT6rqzmhWLt5EpBuApwBcDaAsgFMBLIuhvCqwPv83YH0ujQH8EPuSHtnbaGFqdAu7HgDeVtV/VHUXrC/vpSJSWkQyAXQG8JiqHlDVOQA+A3CNT1lZAOar6gy1rtl6D0AVANVgfVlWq+paAD/B+lJBRC62X4/zI5XjojmsHyLPqWqOqo4D8BusHynR6gHrh9MeVV0B64eJ3+cZMVV9RVXHAtgXJvkqWHU8X1W3AngEQD/A6gKF9UNosKruVdVRAP6A/w6/L4BHVHWrqi4E8GZeWbA25HGquh3ANAANxeo1uQfAfbGuY6zE6j68CMADqrpLVScC+Aqx1Wd+n0dUVHW9qr4K6zMs6DwvhLUNfqqq+2DtgNuKSPPQQiKo+zKqOs/e7vcDqCwiHQA0UNVPYlnHOHsIwMOqOllVc1V1taqujqG8OwB8r6of2EfwO+3POSZH+jZamBrdJ0Rkk4j8JmaXaj0R2SYi9QpYXn+7a2CGiPh96AUh9p93OgPWLyXxvOZNb+1T1ncAUkXkePtX8TUAZgNYB2AjrI22DoBuAObbjfogWL++CiPxec1Zf7sOTw6TL9Jy8/s846UVrF/ZeeYAqC4ile20ZSFHLXPs1w0iUhHWj5DQsvLyzgPQTUQqAGgPYAGsncfzqrotTusSi6YAclR1iec1Z/kLuk1G8HnEXQTzNOra7jL9y2eZDlf3G0SkrYi0hXX0vRXW0e8tcViVuLD3M+0BVBVr+GeViLwsIqU8eQq6jZ4AYItYQ14bROTrKPbTBVXkt9HC0uj+B9YRXW0AQwF8LSKNAEBVV6pqBVVdWYDyXoTVGFaD1X00XEQ6xriM3wG4zh4zKG8vMwCUtiv5NwAPiEhJETkW1q+r0j5l7QQwCsBEANkABgO4QS25AP4N60j5LgDXA3gYVld2G3u84nsRSXQDVBCLAGwAMFBE0kXkDACd4Fl/uw4LciLQGAD3iEhZEWkM64eJ3+cZL5kAtnum8+KyYdLy0sviUJkh7w/N+wSAUwD8AqsbNx3AUbC+9x/aY2QDol2JOMh3XaPYJg/3eSTC4eZZ0PrML++NAF6Ate/qA2v7HQugpL2t/iwinaJZiTiqDut7djGs797RAI6B9WMeQFTbaB1YR563AqgHa3jto3gtsI8iv40WikZXVafYXRPZqvourAbs7BjKm6mqm1X1oKr+D9ZJCxf65RfzhCu/X2rDYH2hxgOYD+Bn+/VV9v/esLok/gHwmj3PVQjvOliNSCsAJQBcCeAbEallL/9YVT1BVTvB+uXcHsBwAO/D6v54BMBb+X0GQVLVAwB6whrLXgfgTgCfwH/9I3ELgL0AlsIaL/4ov/JE5DtPHfaOcp67AJTzTOfFO8Ok5aWHG6/bFfJ+I6+qblHVS1W1Layd9Uuwxu/vgfULuyuAG0WkZZTrEauCrGuk5eWVEVF5EW6TscyzoPXpm1dVZ6tqZ1U9HtYR0TWwTiB6C1aX7tUA3heRcD1CQdlr/39JVdeq6iYAzyKG/axd5mhVnWZ30T8E4CT7oOQQ3EYthaLRDUMRvssyIeWFnHAV9te7PQYyWFWzVLUOrIZ3tf0HVf1bVc9V1ar2xlcZwFSfWbYF8LVaZ23mqnU25loAJ3kz2Rvpy7AaoCoAUlX1b1hjDEdFvvqJp6pzVbWTqlZW1e6wei781j+S8raoam9VraGqrWB9V33LU9WzPHUY7Zmh82HVTZ62ANar6mY7raF4zkS10+eHWZatsOoztKxD8sI66Wyyqs4D0AbAdFXdD2ssKlm9GUsApIlIE89rfst/WAX8PPLec9htMsZ5GnVtj2M38lmmiOse1gmAg1R1L9z6XAHrSMnvhJ6Esz+PVbD2hfEyN6S8vDjsvpbbqCXpja5Yp9V3t7tl0+xfQKcC+D6GMi8WkUwRSbG7Oq+EdSJILMtZSaxLdsT+dfMsrJMScu30FnZXaAkRuRLAGXaecKYBOEdEGtrldYM1jjYvJN91AGap6mwAmwGUsufdBTGcdZgIInKUXYelReQuADVhHZ1HW14jEaks1hndZ8H64sd8XaNdPyVh7RjS7WXO2w7eA3CtiLS0x3wGwV4He3xzNoDB9nsugPXDZ5TPrN4DMEhEKop1cs71CPk8RKQagJtgncQDWN1zXewx/PZIUh3b45ufA3hYRMrYQzPnw+ppidZhP49o2HWZd8lIhj0dyTxHA2gtIhfZ73kQwFxVXRQ6j0jr3t6OS6rqN/ZLywGcJtZZ3xmwtuFkegfAzSJSzf5+3wbrioNYyrtARI4WkXRYQ3kTYx3zPOK3UVVN6h+sX3/TYB3WbwMwGUA3T3o9WF0B9XzePxzAoyGvTYDVP78D1uD4ZWHeNx7AdT5ldgawKuS1pgAWA9gD4G9Yl+5402+DdRLUblhjte1D0ncBOMWOBdY47Up7vRcC6BOSvwqsRric57XesLpvVwDoEun6BFSP/4V1AskuWOPfjf3WP8x7hwAYEfLaJQDW2J/3bADdI3lfSLqGWY7x9uvev86e9DsArLe/O+8AyPCkZdnv32t/F7qG1M18z3QGrCGJHXZ5d4RZvvcA9PJM1wUwxf4cnwnJ2w/WDi2o+qwE4Av7+7wSwBWetGi2yUg+D9/vMMJsk546Nv4inSesLsJFdn2OB5DlSXsdwOuR1L1nXrMB1Pe8djqsbXUtQvZB4T6jAOo0HcCrsPaz62Cd+1LSk16gbdR+/d+wevu2AvgaQN1I3hdSf8VqGw2swhP4RXrT/rL8FWH+JvaXbg+Afj55TrUrbRvC7OwL018k61PY/2D9Wt1tr0eZCN+z2K73Yfnk2Qfrx9cjyV7HGD+fH2H9OBub7GWJcHmL9TZ5mHXNsNdhN6xLW5K+TBEuN7fR/Nc14m2Uj/YjIiIKSNLHdImIiIoLNrpEREQBYaNLREQUkLQgZ9YtpRcHkJPkx9xP435hPuszeRJRnwDrNJm4jR5Z/OqTR7pEREQBYaNLREQUEDa6REREAWGjS0REFBA2ukRERAFho0tERBQQNrpEREQBYaNLREQUkEBvjkFERMVPSunSTtxu0k4jbXDV2U58xoILnbhEt78Tv2BJwCNdIiKigLDRJSIiCggbXSIiooBwTDcB0mpUd+L9TWpF9J70JauN6cX3NnTiCgvc+2ZXWrjPyJcyYVY0i0hUZOzr0cGYLvXdTCfW9i2dePl5ZYx8p5z2hxNPGNfGt/yav+c4ccmvp0a9nGTyjuMuGdrMib+oOtTIl+uJ/5lT04kbgWO6REREFAM2ukRERAFh93KUtl95ghNvPtvs8r3nmDFO3Lfc/yIq7+3t9YzpC8uOduKKvUr6vu/c2u0iKp+osEutUtmJc0aWcuKPmzxr5Fufk+7E5VPGO3G9tNLwddWvvkkbrtzjxGteLGGk/evxW5248pu/+5dPh1h2f1snXtDlRSfuvewsI9/mxxo4caMxkxO/YEnGI10iIqKAsNElIiIKCLuXQ6S0beHEi252z4accMbzRr6qqdPc98Tht8u15VeGvOLfpUx0JFrygjvEsrj5254Us9u4Wqobv7qtqRPP3GkO0azaXcF3XqninjP7bbOvw5YNACMH/deJb1w4wEhLmTgb5G9/tYNhX587oYkx3WBM8eq255EuERFRQNjoEhERBYSNLhERUUA4phtid4OyTrzkrNc8KaUOzRyj17e5d5364O/joiqjPP6M1+Ic8VKOdu9etK+GefeiFT3du35d3GGakXZA3YG+n993745U85ftRj6dNT8uy1lc6IltjemRJ73hmXJ3TWP2mmO6Tw68yonLzt/kJmzcYuRL2fqP/7xT3Dpt+kx/J15wyUtGvkbpmU68d9AOI618P/fOcwfXrfedV3GVnrnfiXfmunG9H7OTsTiFBo90iYiIAsJGl4iIKCBHbPdyWp3axvTC/9Rx4uqT3K7Ech+Zd0BJyVYnXnLA7RL556B5+UHdtG1O3G/eVUba1oXunXWqT3PLqzDJ7O7SXbucuPw2dhPHg3Y82phedpMbf3jim07crkTItSGRGujeEH/vXfuNpKHb3O7rV+d0MtKaXLvQiXP3mXcwK64OlDfv/nR0CXd3lAt3uxn4zjVGvrqjJzlxDqKU676z8e3uPqBFCfOyoLnnv+DEv7T5zEjr2NXtli4/gt3LqY0bGNPzTx3mxLeuOd3N9/NMFGc80iUiIgoIG10iIqKAsNElIiIKyBE1pptaobwTd/h2uZH2RZWvnLjjdHPcxivjO/dykYHn9HPinPmLzXm1cG9lVmnxX0ZapdwlYcsOf1M0ikbuye7Y7Qp3aA3fdnzFyNcozXuplzuO++Ne8xKw+xb0dOJtK83x+3k93ctIHljvPl3q6RrTjXxtS7kP3X62w0gj7d7b+zlxnScmgYCckuKbdtSkfk5c77HgPq8mN00xpr/p6j5UvVfmZiNt23m7nbj8iMQuV1GweIj/bTeDlH2We/nlzrr+TVzVGeYlYDojmEv+eKRLREQUEDa6REREASnS3cspJc0n8WR/5nYv31dlnJHW7HO3D7L5aLcbIb9LDkK7lI20hUsjXEqKh2UfmpcCfeB7+Y/ZbXz58m5OPG2Re0lD81sXGvmq7nbrumrIvG9s19WJN9xS34lvf8287GhQ9fFOPGFvTSNt9gC3i7rniPOd+OA/q1BcNbvXvzsvdUZZ37Qg3T/NHXbo1eVtI+2mVr868TeoGNgyFVbPHT/SN+23D4914hqIfbjgrw+OMaZfOP4jJ25TYqITV0/N8C3jzwPmgN/5n93uxI3umhyaPW54pEtERBQQNrpEREQBKXLdy6kV3W6cRY80NdIWt3jViWeE3FO7+cPLnDhnh3nWGhUOKWXMhxAsfbiNEy/sZJ6VnOI5E3ma5y5ivb+8ycjX7CG3G7npNvds41xErk3Z1U78Y5rbRT39v+2MfJWfdc987VlmG0z+Z+oWJylHNXfizhV+NNKWHHDv1FVl7oHAlik/FX/xDGF1Sd5yFFap5co5cZkUc6f7w153e67xXGRdypLu3qVsf5ejjLT7X3vHiU8tOcNISxd3fzA12+1S7ruol5HvjgY/OPF5ZfYYaa/2dIcPnh92gRPnLAh/NUq0eKRLREQUEDa6REREAWGjS0REFJAiN6a75soWTrz4AvOB01/tdsd73z63m5GWs9G8axQVPtvOa2NMj+v1f06cAvNB5mP3uuM2T/Z3n/LU+AfzVP9In0Ijae6mkNKskZH21heVnPi/773rxG1KbAgpxRO6BRQAACAASURBVF3GVDF/z7aZcoUT195QfL+LS69y71p0WeZGI+3kuX2cuNz/poEKv+W3tXbik0uONdJa/tzXiRtjlm8Z3qcTLb6puhMvuOSlcNkBAGP3ZhrT/b/v58TNX9jkxBlLzG3tFbjnAb00tq6R9k3zz534iXru5aclFvguRlR4pEtERBQQNrpEREQBKXLdyzuP3+ub9sJy90HJpZYU3y68okpDniu/T/0vs9mZ6955at3x7mUGey/sYORr3GRt2Pdv32fezaxXfffB2jdVeN9Im77fLb9jhvdiI7PL2+u3feZFSbUfdddFs7NDsxcbt5/1rRN7LxECgBKvVPZMcfstCuQo/8sv0/8q5Zvm5X1QwqIu7qWBoZf19V52lhPvuLu2kdbkd/dyvUiHlP5cVsN8oXn4fPHGI10iIqKAsNElIiIKSJHrXv6o41DPlPmb4bOW7kMtT3z2TiOtwVf7nTh1/ExQ4VPxS/MG+Df07e3EI5qbDyw9r4x7F6qL/u3eiSxH/e81la3uDc4zJL+vvplmdim7DoZ0ZHWee5kTV7rJTNNlwTyrsyh5Y/OpxnTJb6YmaUkoWs2rrS/we6RdK2N69MmveabSnajV+BuMfE2ude8uJ/vmFHi+h/PgBvc5vCXH/+HEBbl7XSR4pEtERBQQNrpEREQBYaNLREQUkCI3ptshw+3zP6DmuFnFFPcykEWXmk+lOXCJm7f12BuduPw089KRXXXcscJy7oOJUGXubt9l2nSU+XSc6uPdOxXl8NKliOXu3GlMZ5zhTt9Q/UIjbeGQLCc+o507/rJkezUj39+rqzhxagn3O3Bes7lGvqdrTEdBtfzZHHNqdqf7NKKD60PvVlU8pVYob0yXTVmVpCWhRKhT2n2aVkroMZwowllyi/lg+Rbp7j693bQrnbhRb/MuVvEeW03P3G9M7z7oLlfuvn2h2eOGR7pEREQBYaNLREQUkCLXvdzg6+udeMm5r0f8Pu9Djhd3fdNN6BqXxTJMvce9+9BtCzyXkZwb34chFyc5Id21Tf/tTq/wvF4Cfxv5moRM5/lhdEtjOr/u5RUH3Ydd93zpbrfs581LXHIOHgSZVl1rXh7Su+zPTjxzd1bAS1Nw2Wdv903bk1vCN624yFX3uC03tAPY545yNatvM6a972tZ1b0EaWscli+U9+EK808dZqSdOvcSJy6XwDui8UiXiIgoIGx0iYiIAsJGl4iIKCBFbky32U3uaeTdPzUv2ej78tdOXDrFfJLLuaXdB2Z7x3cToUOGe6r8xGM+cOJW/73FyNdo4O8JXQ4yLX/8RCeeedxzIan+43MXP+2O49Z6ZZITh78ggoqyg6e1M6Y/PuZlz5R5qcvop9ynmpXH5EQu1hGlwrXm5ThTJriXDL1cz92Hn/jUXUa+pi+652ccXL0mqnm3GOmWsT7HfGJdyRcqeaY4pktERFTksdElIiIKSJHrXlbPZRnpP80w0j5qXsv3fS9e7F66k5Punsp+0l3mZR9P1pgW6yIavHdpqdM2/APVKXHWDDzJib/v/bQTlxL/B9C/sLWxMV3jndlOHO+74lDyebuUt9xq3nmuebrbpdx/dUcjrcJI92llxWWowXvJDQCcWn5cgcsI7Rp+qmtPJ247yr0N4LwrXzTy9e/UxYnXnlPJSMvZvMWJt/Vxh5FOvm2Kke/B6r85cbuPze7rRmOCGSLgkS4REVFA2OgSEREFpMh1L0erzGdTwr7+ddsTjekn+7jdy3vUvSF2u1//beSr/5Z7BvSmW/YYadOPMx+4TsE5cEZ7Y/qLAW6Xcr00/y7llZ67Tn31n9ONtIw98R1yKE7KrTAfSuK9u1cySZq769t2u/tgjenHfmzk+3FvKSde8oB5d60SBwr+kIyiLufP5cb0x+s6OPEFjcYYafVPXunEqeXKuWXs2GHkO7hshRPPOMY9Djy1j3m1R6W57p2spMoBI235y3WdeP6p7hnnoWcoe7uUG92VnDPOeaRLREQUEDa6REREAWGjS0REFJBiM6brp9735p2r0McNS4t7l6KFnd42s9Xv5sT/y/o+pNTwv2VWrjNPc29iPB+H4mHFuebdxrJ8xnHX5phji31vu9OJS38bfvyfCq7MKPOzHPNICyduVHKjkba0TmsnPrhqdczzzj35aCde3t9Mu6iFexnY49XMcVyvx++6yolLfT/VN19xte86d6z22VHNjbRvmn/pxLeOdS+3mvq6eR5N5prwT+faeJx5gd5xt7iXEz1Ta6KR5r00c+j2LCce/n/nGvkaDUv+XQB5pEtERBQQNrpEREQBKfbdy+nTlxrTJ8y83IknH/uR7/vez/rRM2X+dslW93T2cz0PsW9+i3kTbfNiCopWamW3237Whc+HpGYgnM4TBxjTjUazSzlo/SuYl5+s/8btqpy+pV7M5T/ZYKgTH13Cf1c3Y7+7JfaZeq2R1mjcIifm9nqonCXuPu3X881Lqip+697d67laE9yEhyfAj7ebOLcA939rPfFqJ258xyYnrrQ6+d3JoXikS0REFBA2ukRERAFho0tERBSQYj+mm7tzpzFd4+aKTtxj2HlOfF/Wt0a+EzPcEZ5Ru6oYaff/71Inbny7e6sxjgnFT2pFt55um+KOEWVK+DFcAHhqs3u5SpPrzbF8Pj0oGN5LODbc+quR9lDVOe6EN46au3s7GLL1zXHv8IorR7q3G2xwjzkGyG02ct7bOQLAF53dS8BevNp9ktDuBuYtHL8/0z0Po/v3t7kJ+Ty6qdlb+4zprGlz3eWIZGGTiEe6REREAWGjS0REFJBi370c6uAK98kYOM0Nb7nFvKXNzuPcp1c0H7TJSGv8d3KeXlGcbDrPvfvNGaV/duKcfLqk/vdQZycus5uXCCVDJc8dgab92tRIe/YLt8vwjopm9380mv9yjROX+MO8M1mdJyY5cQMUvstKjgQ56zc4ce0nN/jmuxnu3aqaIrIneuWzmRd6PNIlIiIKCBtdIiKigLB7OULVX5xkTnviwn623JHoort+cuIc9T/3uPHXNzpx01HsUi5MQh+I/lPrsm6MY2MuvyFmHz4TUcB4pEtERBQQNrpEREQBYaNLREQUEI7pUpHUtpR7aVequL8dJ+8z7yHU8mn3UgWOvRNRsvFIl4iIKCBsdImIiALC7mUqkm77wH3Y+KLrX3Xia4bdbOSru8y81IuIKJl4pEtERBQQNrpEREQBYaNLREQUEI7pUpFUf7A7Vtt98NFOXBccwyWiwotHukRERAFho0tERBQQUS3KjwMmIiIqOnikS0REFBA2ukRERAFho0tERBQQNrpEREQBKXKNrogMEZEDIrJLRMpE+J6/RGS/iIzIJ4+KyG4ReSx+Sxs8ERknIvtEZGKylyVSIjLcrp8VEeZvatd/johc55Ons4jk2vnOjOsCB0hEMux1OCAijyZ7eSLBbTR/RXEb9Sru26uIdLWXM1dEuhb0/UlpdEWkhf3F2y4if4rIBQUsYqSqZqrqbru8CiLyrohssP+GeDOraiMAj0dQbltVvd+znENFZLH94fYLsx63i8g6ez2GiUiGJy1LRH4WkT0isii/yrF3rMNEZIdd3h2etLoiMllEtojIMyHvGyMi7UPW9TQAN0awrnElIpVEZLS9U/xbRK4oYBFPq2qWpzzfz0RVl6hqJoAJhylzjf09GWOXWVNEvhKRNfYOPMubOb952umn23W5x67b+n4zzq/+7XKWi8haEbnU83oFEZkpImU965ptr+sHh1nXuBKRASIyXUSyRWR4FEWEbqNd7M9je7idNbfRYInIeLvh32X/LS5gEaHba15DvMvzlwrEtL2KiNwvIivtz/1jESnnmWdtEfnS/txXiYjvZxpBWQNFZJOIzBOR1p7XO4rIF96yVPUne31WIgqBN7oikgbgSwDfAKgE4AYAI0SkaQzFPgegNIAsAB0A9BGRq2NcVACYA6A/gJmhCSLSHcA9AE6359sQwEOeLB8BmAWgMoD7AXwmIlV95jMEQBMA9QF0AXC3uL/27gXwLoAGAHrmbcD2znqZqk6PfvXi6hUA+wFUB9AbwGsi0iqG8obA/zOJVi6AMQAuKug8RaQKgM8BPADrezsdwMh85pVf/T8PoAeAM2F9Tqn2608AeFJVd0azcnG2BsCjAIbFqbzddlkD41ReHm6j0RtgN3KZqtosDuU97SkvU1VzYiyvL4A+ADoCqAWgFICXPOkjACyHtc85B8DjItKloGWJSE0A18L6frwO4En79TQAzwC4Lcb1MCTjSLc5rJV+TlVzVHUcgN9gfSDR6gGrwveo6goAbwO4JtYFVdVXVHUsgH1hkq8C8LaqzlfVrQAeAdAPsLpTABwLYLCq7lXVUQD+gP/Ovi+AR1R1q6ouBPBmXlmwNuRxqrodwDQADe1faPcAuC/WdYwHsboQLwLwgKruUtWJAL5CbHWa32cSFf3/9u48TIri/uP4p7iWUwQSIMopiIByCXgGRNR4PEok0USUJJgoISGaX1RiEo0QNI/JYw6VeEVFo4kXiYjkiVcUcikiyK2QyCUYBYIih8i19fuje6u7xp1lZmemFnbfr+fZ5/n2VE1Nz/T21FRVV5e1G6y1dyr6HPN9zS9IWmatnWat/VjRl3A/Y0zPzEJyOP7NrLVLrbWLFP1QaWOMOU5SV2vtE4W8x2Kx1j5prX1K0uYilTfXWvuwpFXFKC9VLudo7XWeouO3zlq7XdLPJX3ZGNPUGNNc0jBJP7XW7onPpT8q+/d+1rIkdZK0wFq7VdJfFVW+UlTZPh3XKUVTE5WuyfJYukm/xRjz2QLK9corkaMV/cqusEhSO2NMmzhtVUaLZVH8uMcY00rRj5DMsiryLpV0hjHmUEmDJL2h6MvjVmvtliK9l0L1kLTPWvvv1GPuPRhjOsXHtFMuheXwmRRdDq/pHe+423Rlln3a3/HfaIzpZ4zpp6j1/YGi1u+VRXgrQVTzHA2Nc7RqN8ddqv8yxgyreDDf8zXl23FX73xjTLYfL/kw+uT3epmiHgeTeiydnu17v6qy3pLUJz5+p0taZozpKOkiSb8o8D18Qk1UusslbZQ0wRjT0BjzOUmnKOoeliRZaw+NW0u5elbSD4wxLYwx3RX92mm6n+cUqrmkD1PbFXGLStIq0lvok5pnPD8z782Shkj6m6Iu3IaS+kqaaYx5xBjzd2PMd6r7JoqkyvdrrX07Pqa5joHs7zMphf29Zr7HtKq84yTdJum3inoDviXpRUmNjTHPxeOMp1TnTYRSjXO0JnCOZnetohbd4Yr+D2caY7pJ1TpfJel2RRVYW0VDMA8aY04ucB+fkXSZicbeW8b7LElN4x9L/5L0Y2NMY2PMsYp6KbJ971dV1mZJP5X0kqJu6msUnZ/XShppjPlbPHbcocD3I6kGKl1r7R5J5yt6c+9JulrSE5LWF1DslZJ2SvqPovHiR6sqzxjzTGqw/5JqvuZ2SYektivibZWkVaRXNla3PeP5Xl5r7fvW2i9ba/sp+keYIukKRV1XSxX9MhtnjOldzfdRDPm831zLqygjp/IyLuDI9xd6Lq+Z7zHNmtdau9BaO8xae7yiVtHXFV1EdJ+iMcdLJT1sjKmsV6hO4BwtLWvtq9babfGFer9TVIGdU0B5r1trN1tr91pr/6Lowr8vZMuf4/k6VdF3+WxJyyTNih+v+G6/RFHX/jpJd8Wvme17v8qyrLWPWmuPtdaerai1vEvReP8vFHVNT1ORWr01cvWytXaxtfYUa20ba+2Zin5xzS2gvPettZdYa9tba49W9L6ylmetPTs12F/dq0KXSeqX2u4naUP8q2mZonGdFhnpyyrZlw8kvVtJWZ/Iq+iisznW2qWS+kiaZ63drWgsqtTd6VX5t6QGxpgjU49lew/7lednUvGc9AUceV9VmMNresc7HsfulmWfcj7+ii4CvN5au1PJMV2jqLWU7aKeWo9zNDiryof+SlJeLuertbbcWjvRWtvFWttB0ef9Tvwna+1aa+251tpPxz9g2yjL9/7+yqpgjGmi6Afw1Ypa7uvisd7XFPVeFKympgz1jbsEmhpjrpH0GUkPFlBeN2NMG2NMfWPM2Yr+8Que02iMaWSMaazon6dhvM8Vn9lDkr5hjOkdj/lcr/g9xGObCyVNjJ8zUtEB+1OWl3pI0vXGmFbxhTmXK+PzMMa0lTRe0QU8UnTV3qnxBQWDVOQLVPIRj28+KWmyMaZZ3K30eUkPF1Dsfj+T6oiPZ8W0kbJ4O5fXnC7pGGPMF+Pn3CBpsbV2eeZr5Hr8jTFnSGpsrf1z/NBqScNNdNV3mYp0EVN1GGMaxO+zvqT68fuo9vrbxph6cXkNo03T2BjTqAj7yTmaJxNNTTuz4pjGPQlDJT1XQJkXGGOax8f5c5JGK7qYspD9bB1/t5u4l+BXkiZba8vj9F7xkGIjY8xoSZ+L8+RdVsr1kh601v5X0ZSgo4wx7RRdsV6c42etDf4n6RZFF49sV9TX3j0jfbukIVmeO0nS7zMe+5KiKQ4fKTqRzszleRnptpL9mB0/nv4blkq/StIGSVslPSCpLJXWJX7+TkkrJJ2eSrtE0ZWwFdtliro/tsblXVXJ/j0k6cLUdkdJr8af4y8z8o6R9M/Ax7S1pKcUTQ15W9LFqbRO8THtlOW5D0q6KeOxXD6T2ZIuy1LmMEnrsxxn7y/X11TUTbg8PqazJXVJpd0t6e5cjn/qtRZK6px67DRJaxS1qi7a32dU4uM5qZLPalIqPd9zdFgl5c3e3/MqOXaco4Uf208rarltk7RF0hxJZ6TSq3O+/kPROPdWRReZXVTJ82Yrj/NV0QWaKxR9r6/N/MwVXV28SdF3zj8lDcpId/+j+ysrznNU/Lk0SD02QdL/FA0D9cnIv0YZ53VOn39NHPQC/2Gujz/kLYqmXuTynBXxAZhaRZ6P43+aG2v6PRb4+bwQn0wv1vS+5LHP98bHZ2WO+Y+Mj/9HksZkyTM0/jLdokp+hB0sf/GX/Zb4f35iTe9PjvvMOVr1ez3oztGM/a/T56uiH8db4v09Nd/ns54uAACBHHT3XgYA4GBFpQsAQCDVvhqxOs6odyF92TXkhfJpRZ/zyfGsOaU4nhLHtCZxjtYu2Y4nLV0AAAKh0gUAIBAqXQAAAqHSBQAgECpdAAACodIFACAQKl0AAAKh0gUAIBAqXQAAAqHSBQAgECpdAAACodIFACAQKl0AAAIJusoQABTTW78+wcUrv3y3l/bVtUNdvOHErcH2CfnZO3ygi1ePTKqkq0/7i5dvbMs1Lq4nfwGfciWLKU3cOMDFM9cc4+U77Ob6ycbcJdXa30LR0gUAIBAqXQAAAqF7GbVag/btXPzhyV1c/M4Z/treq0f81sV77D4v7eSFF7l407pWLu79s/e8fHvXvF3QviJ/J5/wRta0hzr/3cVDRn7TS2s6/dWS7VNd9c61J3nbO47c7eJRA+dmfd5P2ibnXrnKXVwvo02YTus1e6yX1vbpMhe3eHyOiw9T9v+PmkJLFwCAQKh0AQAIhO5lHPRMWdK1tOonx3ppv7ngPhef0uSjrGXsscnvz3Q3liT9o/8jyUb/VNjm616+ThfmtLsoonQXclX+O9S/2rX79FLsTd226MrfeNvpK4o37Nvp4js3+93QPZ5Juv6b/aeRixv/zx8CanP/Ky7upgWF7WwNoqULAEAgVLoAAARCpQsAQCCM6WbYNywZE2xwwwYXzzzqaS9fQ5Pc2aSqKSZtrmvoYrPmHS/f5vN6u7j1U0u9tPJt2/LZ7Trt7QnJHW2WfOW2apVx6drTXHx/5xdyes7Ck6Z62yM0uFqvjdLr/r05+8+EggxdcoG3/VKfx12cHsedP8Bv6/XQvNLu2AGGli4AAIFQ6QIAEEid7F5OTzHZNqK/lzbx5qTLMD3FxJ9EIu1JXc1e1RSTY388xsX92vu/cWZ0SS6xH3zoFV5auykvV77zkCTZE/u5eOrXp+T9/L4PXOltd73xdRf3/PV4L2355+/Iu3ygrjn08t3e9p9fbOPi8w+d7+KFvS728u178z+l3bEDDC1dAAACodIFACAQKl0AAAKpk2O6u4b1cfFLt/4ma75ZO5u7+Iab/Fv+NfzIZmZ3tnZOfss0St158PvX+FNMPizf6+Lm7/rTjuBLj+FKkr3pfRcPTIboPzH2Pn17WxdPHTPCxV1e9Vc9seXJ53/U9xZ5aWc/9S0X33h3siLKoDL/mJ2+NJnm9ddjWmS+BZRAt8fHuThzEfu09GL3ElOISmHvuvXe9g+mX+LiN0Yn37O72/vnRv03S7tfBxpaugAABEKlCwBAIHWmezndPXnzXfdkzTdq5Tku3jqxo4tbzXqlsuyVatm9q4v7T1vp4l6N/N84PWd8z8U9/sii2lXZOLiZt/1az6SrPn13sA/L/WkLE59I7g7W5ZXcjqHdtcvbbvh8csec0c8l3ZnLzvOHJia0To71vY9+zUvrOsrvskZxVNWljBqWWtipXmpj89GNvWytzUDlomxeMrVo39athe1bDaKlCwBAIFS6AAAEUme6lz+4LllEOX216znLv+Dlq3/NIUm84HVVx5aB7Vw8se0TWfN1fL5axddJ9U7f7G2n7wKWvjvYpatGePm6/Dj3YYFc9PhWctXzlM8e7aVd1Xq5iy/p/ZqX9rIaCajNGnTs4G3/7Pw/uDi9oP2cH/qLktRLtf3S53W9jDbhsCUXunjXNP/cSy9wf6CjpQsAQCBUugAABEKlCwBAILV2THf1Y3297WUDHnDx+r3J+G6961p5+eyCxXm/VnrVIknq/n9vJOWnftekF0qXpCZP+XdFgq/B4Ye5+Oqj/prTc1ZNO9LbbqdNRd2ntKkzTve2r7p0eZacQO2UHsc95zl/WtyIZh+4eOLGAS6eueYYL5+dc2ilZY+46J/e9lVHJN8B50/e4qWVT07GjM/6ylgXp6cZSQfGVCNaugAABEKlCwBAILW2e/mrvf2u2/Sl6Gv3JtOCNCf/7mTJ71Jecat/M/4ZnZJFz9M34F97y1FevqbiLlRV+eCznVx8QfMZWfONXTfMxYen7gAmSXtVM45p4t/8fe4Rw128d9WawHsDlMb2/skQ0NiW/jk6dPGXXHzI2cl5eZjeUC7m/9xvEy7qMMTF11/W2Us74awlLn724WRRkju2dPPyPXNpUobmLlFNoKULAEAgVLoAAARSa7uXi63+0X7X8JtXtHTx8vPuyMzupNfkbfHyai+NFXSrtulYs/9Mklb+rJeLm7x3YFwRfm4z/w5avxrU3sXN6V4OjvVzS6PxzOR8O3emv3DBIVqZmb0ge9e/4+JOk97x0v47KYkHXHuFizOvgL7x8WShlB9+Y5yX1uCl+UXYy/2jpQsAQCBUugAABEKlCwBAILV2TPdPq/t72xPaJJeHDyjb4eIhiz/Oqbzjmj7pbZ/aJHleeWbmlKsXXeDiDhuW5fRaiOxrmn3FkbQD5c5eDU19F6dXPgIQzuE/f9nFi/7Q0Uv7zHMfunjyffd6ad/96XgXl3LVIlq6AAAEQqULAEAgtbZ7uf1o/5LyEU+NdPGfeyZ3Tkl3O+djSOqy9PJR/vSQf/R/xMVt721arfIh9e27xsXlVXbiHxj22GQS2MGwv0Btl55mJEnTfnSmi9+d5E8ju/P62138tY7fdXGnSS+rmGjpAgAQCJUuAACBUOkCABBIrR3TLd+2zX/gtGR7+Mhvu3jjwOy/O1q9mcz7aPkHv/9/08O7XLy8/2Ne2v0fdnFx02XvurimVrxBeGv37va2m2zanSUngFCazEimFy6an3060cLLb3PxiEmDi7oPtHQBAAiEShcAgEBqbfdyVZpOTxaP7zK9emUsH36fizOnh9yx4hQXH7YutwWbcfC57Pzns6Z9/oEJ3nanWcWddoDIV9cOdfFDnf+eNd9bvz7B22bVIWROJ7p90akuHnfKqpK9Li1dAAACodIFACCQOtm9XB2Zi9hLyYLHmVeqtru9cYA9qv123HCYi+c9UN9LG1SW3P3p7Wl9XNzpwurdYaw6BjdZ7W3P3WVc3OWWRV4a96cCDjDH9fE2Hz7hfhffsaVbyV6Wli4AAIFQ6QIAEAiVLgAAgTCmm6NVExtlTbtwwWXedvtZr5d6d+qEen9b4OLxt37HS3vt2ikufuH4u1w85tQrvXz1i3wsVj/W18UnN57vpZ20YJSLW+/4d1FfF4mPRh7v4oc631ODe4K0tT85ydtu/L8kbjflwJgyV793DxdvnbzDS+vQYKeLnx0zJJVS3OtEaOkCABAIlS4AAIHQvVwFe2I/Fz99/J0Zqcm0IPNiq0B7VHd9Zvb73vag4aNdPG/w7128fpg/XavzrMJfe8cXk+7MJ45PFrp+ZVeZl6/1TUwVC6Hr99+s6V1AbPM3TnTxksumeGm9ZifDbu38pII16NjB2157cadK8x1xjn9nqR91fNTFc3b604JGTkruItf6tVcK3cWsaOkCABAIlS4AAIFQ6QIAEAhjulXYOLiZi7s28Mfr0isLNfjYCqVVvni5t334dcltOadPb+3ip8fc4uU761NXufjI8a8qGzPwaBdvOLGll3bP1cmC1r0aJb9Te84c6+XrMWeuUHzpKUJS7tOEhoz/pou7T2dVoVJraPxbtb45LFmJbcHq5Pvy4lcu9/KZVDz0iLdcvGJLWy/frD7TXFxP/lTActlUWlLinVu6evlGvZT8T/Se9K6X1np96cZx02jpAgAQCJUuAACB0L1chY8/lXRZZC5Uf+v7vV3c5t4w3RJI7Fu2wsW/OytZfPqe3/rH6dlzf+XiJ4YMdPFjjwz38t03NpnTMKAs+5pAZ71xgYt73rXNS2MlofC6PT7OxZkL0zdV9uEEFEeb+5PvvpN2jPPSNp63q9Ln/O7E+73t48qS79n06j7lXsezPwWpfLN/h8Ajpu+p9LUazX/L2+6xdtelKAAAAZ9JREFUdZ6L91b6jNKjpQsAQCBUugAABEL3chVGn5/9dkZTZ5zu4i6ie7km7V21xsVloz7tpY0b8F0XN7z2PRfPv+I2L1/PmeOzlt/1yaTjuGzWYheX79md974if02n+93EZ07v7+Lu4qrkA0WLx+ZkbFeeb7KOzbFEf/immxZkyZfdvryfUXq0dAEACIRKFwCAQKh0AQAIhDHdKvxpdTJ2NKFNcRcyRmns27TJ2274fGr7+SQcocFevh7K7W5S3HsMQCFo6QIAEAiVLgAAgdC9XAX7YnIj/R918G+63m7egXgxOgDgQEZLFwCAQKh0AQAIhEoXAIBAGNOtQrvbX3bx0tv9tCY5TjEBAKACLV0AAAKh0gUAIBBjLffYAQAgBFq6AAAEQqULAEAgVLoAAARCpQsAQCBUugAABEKlCwBAIFS6AAAEQqULAEAgVLoAAARCpQsAQCBUugAABEKlCwBAIFS6AAAEQqULAEAgVLoAAARCpQsAQCBUugAABEKlCwBAIFS6AAAEQqULAEAgVLoAAARCpQsAQCBUugAABPL/HkJlJsFJTo0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 576x576 with 16 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "with tf.Session() as sess:\n",
    "    ckpt = tf.train.get_checkpoint_state('./')\n",
    "    if ckpt and ckpt.model_checkpoint_path:\n",
    "        saver.restore(sess, ckpt.model_checkpoint_path)\n",
    "        final_pred, acc = sess.run(\n",
    "            [pred, accuracy],\n",
    "            feed_dict={\n",
    "                x: mnist.test.images[:16],\n",
    "                y: mnist.test.labels[:16],\n",
    "                keep_prob: 1.0\n",
    "            })\n",
    "        orders = np.argsort(final_pred)\n",
    "        plt.figure(figsize=(8, 8))\n",
    "        print(acc)\n",
    "        for idx in range(16):\n",
    "            order = orders[idx, :][-1]\n",
    "            prob = final_pred[idx, :][order]\n",
    "            plt.subplot(4, 4, idx + 1)\n",
    "            plt.axis('off')\n",
    "            plt.title('{}: [{}]-[{:.1f}%]'.format(\n",
    "                np.argmax(mnist.test.labels[idx]), order, prob * 100))\n",
    "            plt.imshow(mnist.test.images[idx].reshape((28, 28)))\n",
    "\n",
    "    else:\n",
    "        pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **LeNet-5模型网络结构：**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 输入层：LeNet模型的输入层是一个32x32的单通道图像，即灰度图，比MNIST数据集中最大的字母还要大，便于潜在的明显特征能够出现在最高层特征监测感受野的中心。\n",
    "* C1层：卷积层，一共包含6个特征图。卷积核尺寸5x5，步长为1，padding的方式为VALID，所以特征图像的尺寸为28x28像素。\n",
    "* S2层：为下采样层，也叫池化层，每个单元与C1所对应的特征图上2x2像素的区域连接，这四个输入相加后乘以一个可训练的系数，再加上一个可训练的偏置量后应用Sigmoid函数，2x2像素的感受野不重叠，所以S2的特征图尺寸为14x14像素。**（不太懂：池化层怎么也有biase和激活函数？）**\n",
    "* C3层：卷积层，一共包含16个特征图。因为卷积核的尺寸为 5x5，步长为1，padding的方式为VALID，根据公式可算出，其最后的宽高均为（14-5）/1+1=10像素，最后得到的结果为10x10像素的特征图尺寸。\n",
    "* S4层：池化层，每个单元与C3所对应的特征图上2x2像素的区域连接，这四个输入相加后乘以一个可训练的系数，再加上一个可训练的偏置量后应用Sigmoid函数进行激活，此时2x2像素的感受野不重叠，所以S4的特征图尺寸为5x5像素。\n",
    "* C5层：卷积层，一共包含120个特征图，每个单元连接到S4所有特征图上的5x5区域。因为卷积核尺寸为5x5，步长为1，padding方式为VALID，根据公式算出最后的宽和高均为（5-5）/1 + 1 = 1，得到1x1的特征图尺寸。\n",
    "* F6层：全连接层，共包含84个神经元，将该层的输入与权重做点乘积，然后加上偏置量后应用Sigmoid函数进行压缩。\n",
    "* 输出层：包含了10个神经元，分别对应于10种分类结果。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **本模型网络结构：**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 输入层：输入层是一个28x28的单通道图像，即灰度图。\n",
    "* C1层：卷积层，一共包含6个特征图。卷积核尺寸5x5，步长为1，padding的方式为VALID，所以特征图像的尺寸为24x24像素,relu函数激活。\n",
    "* S2层：为下采样层，也叫池化层，kernel=2x2,步长为2，padding的方式为VALID,所以特征图像的尺寸为12x12像素。\n",
    "* C3层：卷积层，一共包含16个特征图。因为卷积核的尺寸为 5x5，步长为1，padding的方式为VALID，根据公式可算出，其最后的宽高均为（12-5）/1+1=8像素，最后得到的结果为8x8像素的特征图尺寸，relu函数激活。\n",
    "* S4层：池化层，kernel=2x2,步长为2，padding的方式为VALID,所以特征图像的尺寸为4x4像素。\n",
    "* F5层：全连接层，共包含120个神经元，将该层的输入与权重做点乘积，然后加上偏置量后应用relu函数进行激活。\n",
    "* F6层：全连接层，共包含84个神经元，将该层的输入与权重做点乘积，然后加上偏置量后应用relu函数进行激活。然后添加0.6的dropout\n",
    "* 输出层：包含了10个神经元，分别对应于10种分类结果。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 二、卷积带来的特性"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 局部相关性\n",
    "    - 相邻的像素结合起来表达一个特征，距离较远的像素影响较小\n",
    "    - 随着层数的增加，feature map里面的点映射到原图的感受野越来越大\n",
    "    - 不仅仅可以适用于图像，其他具有局部相关性的数据均可以尝试卷积网络处理\n",
    "    - 带来一定的遮挡不变性\n",
    "* 参数共享\n",
    "* 平移宽容\n",
    "    - 在任何位置都可以激活神经元\n",
    "* 缩放宽容\n",
    "    - 一定程度内的图像缩放不会产生太大干扰\n",
    "* 少许降维\n",
    "    - $(H,W)_output = ((H,W)_input - (H,W)kernel + padding*2)/stride + 1$\n",
    "    - 例如:10x10的图像，用5x5的卷积核，采用valid方式(padding=0)，stride=2 输出feature map为: floor((10 – 5 + 0*2)/2) + 1 = 3 x 3\n",
    "* 对输入的尺寸不做要求\n",
    "    - 全连接里面，权重w的尺寸是由输入决定的\n",
    "    - 卷积核的尺寸只与网络设计有关，与输入没有任何关系\n",
    "    - 卷积核固定的情况下，feature的尺寸是随着输入变化的\n",
    "* 局部相关性\n",
    "* 位置不变性\n",
    "* 降低了模型参数，运算量，复杂度（泛化能力）。\n",
    "* CNN适用任意尺度的输入，对图片尺寸依赖度比较小"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 三、为什么卷积的效果要比全连接网络好？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "    1、具备局部相关性，全连接各个输入相互独立；\n",
    "    2、大大降低输入维度，降低计算量，防止过拟合；\n",
    "    3、参数共享，参数量大大减少，降低了计算量"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
